#### Deployment fails — _... build plugin for file suffix "hdbmigrationtable" [8210015]_ {#missingPlugin} | | Explanation | | --- | ---- | | _Root Cause_ | Your project configuration is missing some configuration in your _.hdiconfig_ file. | | _Solution_ | Use `cds add hana` to add the needed configuration to your project. Or maintain the _hdbmigrationtable_ plugin in your _.hdiconfig_ file manually: `"hdbmigrationtable": { "plugin_name": "com.sap.hana.di.table.migration" }` #### Deployment fails — _In USING declarations only main artifacts can be accessed, not sub artifacts of \_ This error occurs if all of the following applies: + You [added native SAP HANA objects](../advanced/hana#add-native-objects) to your CAP model. + You used deploy format `hdbcds`. + You didn't use the default naming mode `plain`. | | Explanation | | --- | ---- | | _Root Cause_ | The name/prefix of the native SAP HANA object collides with a name/prefix in the CAP CDS model. | _Solution_ | Change the name of the native SAP HANA object so that it doesn't start with the name given in the error message and doesn't start with any other prefix that occurs in the CAP CDS model. If you can't change the name of the SAP HANA object, because it already exists, define a synonym for the object. The name of the synonym must follow the naming rule to avoid collisions (root cause). ### How do I pass additional HDI deployment options to the multitenancy tenant deployment of the `cds-mtx` library You can add a subset of the [HDI deploy options](https://help.sap.com/docs/SAP_HANA_PLATFORM/4505d0bdaf4948449b7f7379d24d0f0d/a4bbc2dd8a20442387dc7b706e8d3070.html) using the environment variable `HDI_DEPLOY_OPTIONS`.\ When making use of these parameters, for example `exclude_filter`, please always check if the parameters are consistent with your CDS build configuration to avoid deployment problems. For example, make sure to not exclude generated SAP HANA tables that are needed by generated views. ### How can a table function access the logged in user? The _cds runtime_ sets the session variable `APPLICATIONUSER`. This should always reflect the logged in user. Do not use a `XS_` prefix. ## MTXS ### I get a 401 error when logging in to MTXS through App Router { #mtxs-sidecar-approuter-401} See [How to configure your App Router](../guides/extensibility/customization#app-router) to verify your setup. Also check the [documentation about `cds login`](../guides/extensibility/customization#cds-login). ### When running a tenant upgrade, I get the message 'Extensions exist, but extensibility is disabled.' This message indicates that extensions exist, but the application is not configured for extensibility. To avoid accidental data loss by removing existing extensions from the database, the upgrade is blocked in that case. Please check the [configuration for extensibility](../guides/extensibility/customization#_1-enable-extensibility). ::: danger If data loss is intended, you can disable the check by adding cds.requires.cds.xt.DeploymentService.upgrade.skipExtensionCheck = true to the configuration. ::: ## MTA { #mta} ### Why Does My MTA Build Fail? - Make sure to use the latest version of the [Cloud MTA Build Tool (MBT)](https://sap.github.io/cloud-mta-build-tool/). - Consult the [Cloud MTA Build Tool documentation](https://sap.github.io/cloud-mta-build-tool/usage/) for further information, for example, on the available tool options. ### How Can I Define the Build Order Between MTA Modules? By default the Cloud MTA Build Tool executes module builds in parallel. If you want to enforce a specific build order, for example, because one module build relies on the outcome of another one, check the [Configuring build order](https://sap.github.io/cloud-mta-build-tool/configuration/) section in the tool documentation. ### How Do I Undeploy an MTA? `cf undeploy ` deletes an MTA (use `cf mtas` to find the MTA ID). Use the optional `--delete-services` parameter to also wipe service instances.
**Caution:** This deletes the HDI containers with the application data. ### MTA Build Complains About _package-lock.json_ If the MTA build fails with `The 'npm ci' command can only install with an existing package-lock.json`, this means that such a file is missing in your project. - Check with `cds --version` to have `@sap/cds` >= 5.7.0. - Create the _package-lock.json_ file with a regular [`npm update`](https://docs.npmjs.com/cli/v8/commands/npm-update) command. - If the file was not created, make sure to enable it with `npm config set package-lock true` and repeat the previous command. - _package-lock.json_ should also be added to version control, so make sure that _.gitignore_ does __not__ contain it. The purpose of _package-lock.json_ is to pin your project's dependencies to allow for reproducible builds. [Learn more about dependency management in Node.js.](../node.js/best-practices#dependencies){.learn-more} ### How Can I Reduce the MTA Archive Size During Development? { #reduce-mta-size} You can reduce MTA archive sizes, and thereby speedup deployments, by omitting `node_module` folders. First, add a file `less.mtaext` with the following content: ::: code-group ```yaml [less.mtaext] _schema-version: '3.1' ID: bookshop-small extends: capire.bookshop modules: - name: bookshop-srv build-parameters: ignore: ["node_modules/"] ``` ::: Now you can build the archive with: ```sh mbt build -t gen --mtar mta.tar -e less.mtaext ``` ::: warning This approach is only recommended - For test deployments during _development_. For _production_ deployments, self-contained archives are preferrable. - If all your dependencies are available in _public_ registries like npmjs.org or Maven Central. Dependencies from _corporate_ registries are not resolvable in this mode. :::

## CAP on Cloud Foundry ### How Do I Get Started with SAP Business Technology Platform, Cloud Foundry environment? For a start, create your [Trial Account](https://account.hanatrial.ondemand.com/).

### How Do I Resolve Errors with `cf` Executable? { #cf-cli} #### Installation fails — _mkdir ... The system cannot find the path specified_ This is a known [issue](https://github.com/cloudfoundry/docs-cf-cli/issues/57) on Windows. The fix is to set the `HOMEDRIVE` environment variable to `C:`. In any `cmd` shell session, you can do so with `SET HOMEDRIVE=C:`
Also, make sure to persist the variable for future sessions in the system preferences. See [How do I set my system variables in Windows](https://superuser.com/questions/949560/how-do-i-set-system-environment-variables-in-windows-10) for more details. #### `cf` commands fail — _Error writing config_ This is the same issue as with the installation error above. ### Why Can't My _xs-security.json_ File Be Used to Create an XSUAA Service Instance? { #pws-encoding} | | Explanation | | --- | ---- | | _Root Cause_ | Your file isn't UTF-8 encoded. If you executed `cds compile` with Windows PowerShell, the encoding of your _xs-security.json_ file is wrong. | _Solution_ | Make sure, you execute `cds compile` in a command prompt that encodes in UTF-8 when piping output into a file. [You can find related information on **Stack Overflow**.](https://stackoverflow.com/questions/40098771/changing-powershells-default-output-encoding-to-utf-8){.learn-more} ### How Can I Connect to a Backing Service Container like SAP HANA from My Local Machine? { #cf-connect} Depending on, whether the container host is reachable and whether there's a proxy between your machine and the cloud, one of the following options applies: * CF SSH The second most convenient way is the `cf ssh` capability of Cloud Foundry CLI. You can open an SSH tunnel to the target Cloud Foundry container, if these prerequisites are met: - There's **no HTTP proxy** in the way. Those only let HTTP traffic through. - SSH access is enabled for the CF landscape and your space (in _Canary_ this is true, otherwise check with `cf ssh-enabled`). Use it like this: ```sh cf ssh -L localhost::: ``` where `` has to be a running application that is bound to the service. Example: Connect to a SAP HANA service running on remote host 10.10.10.10, port 30010. ```sh cf ssh -L localhost:30010:10.10.10.10:30010 ``` From then on, use `localhost:30010` instead of the remote address. [Learn more about **cf ssh**.](https://docs.cloudfoundry.org/devguide/deploy-apps/ssh-apps.html){ .learn-more} * Chisel In all other cases, for example, if there's an HTTP proxy between you and the cloud, you can resort to a TCP proxy tool, called _Chisel_. This also applies if the target host isn't reachable on a network level. You need to install _Chisel_ in your target space and that will tunnel TCP traffic over HTTP from your local host to the target (and vice versa). Find [step-by-step instructions here](https://github.com/jpillora/chisel). For example, to connect to an SAP HANA service running on remote host 10.10.10.10, port 30010: ```sh bin/chisel_... client --auth secrets https:// localhost:30010:10.10.10.10:30010 ``` From then on, use `localhost:30010` instead of the remote address. [Learn more about **Chisel**.](https://github.com/jpillora/chisel){ .learn-more} ### Aborted Deployment With the _Create-Service-Push_ Plugin If you're using _manifest.yml_ features that are part of the new Cloud Foundry API, for example, the `buildpacks` property, the `cf create-service-push` command will abort after service creation without pushing the applications to Cloud Foundry. Use `cf create-service-push --push-as-subprocess` to execute `cf push` in a sub-process. [See `cf create-service-push --help` for further CLI details or visit the Create-Service-Push GitHub repository.](https://github.com/dawu415/CF-CLI-Create-Service-Push-Plugin){.learn-more} ### Deployment Crashes With "No space left on device" Error If on deployment to Cloud Foundry, a module crashes with the error message `Cannot mkdir: No space left on device` then the solution is to adjust the space available to that module in the `mta.yaml` file. Adjust the `disk-quota` parameter. ```sh parameters: disk-quota: 512M memory: 256M ``` [Learn more about this error in KBA 3310683](https://userapps.support.sap.com/sap/support/knowledge/en/3310683){.learn-more} ### How Can I Get Logs From My Application in Cloud Foundry? { #cflogs-recent} The SAP BTP cockpit is not meant to analyze a huge amount of logs. You should use the Cloud Foundry CLI. ```sh cf logs --recent ``` ::: tip If you omit the option `--recent`, you can run this command in parallel to your deployment and see the logs as they come in. ::: ### Why do I get "404 Not Found: Requested route does not exist"? In order to send a request to an app, it must be associated with a route. Please see [Cloud Foundry Documentation -> Routes](https://docs.cloudfoundry.org/devguide/deploy-apps/routes-domains.html#routes) for details. As this is done automatically by default, the process is mostly transparent for developers. If you receive an error response `404 Not Found: Requested route ('') does not exist`, this can have two reasons: 1. The route really does not exist or is not bound to an app. You can check this in SAP BTP cockpit either in the app details view or in the list of routes in the Cloud Foundry space. 2. The app (or all app instances, in case of horizontal scale-out) failed the readiness check. Please see [Health Checks](../guides/deployment/health-checks.md) and [Using Cloud Foundry health checks](https://docs.cloudfoundry.org/devguide/deploy-apps/healthchecks.html) for details on how to set up the check. ::: details Troubleshoot using the Cloud Foundry CLI ```sh cf apps # -> list all apps cf app # -> get details on your app, incl. state and routes cf app --guid # -> get your app's guid cf curl "/v3/processes//stats" # -> list of processes (one per app instance) with property "routable" # indicating whether the most recent readiness check was successful ``` See [cf curl](https://cli.cloudfoundry.org/en-US/v7/curl.html) and [The process stats object](https://v3-apidocs.cloudfoundry.org/version/3.184.0/index.html#the-process-stats-object) for details on how to use the CLI. ::: ## CAP on Kyma ### Pack Command Fails with Error `package.json and package-lock.json aren't in sync` To fix this error, run `npm i --package-lock-only` to update your `package-lock.json` file and run the pack command again. > Note: After updating the package-lock.json the specific dependency versions would change, go through the changes and verify them. ::: tip For SAP HANA deployment errors see [The HANA section](#how-do-i-resolve-deployment-errors). ::: ## CAP on Windows Please note that Git Bash on Windows, despite offering a Unix-like environment, may encounter interoperability issues with specific scripts or tools due to its hybrid nature between Windows and Unix systems. When using Windows, we recommend testing and verifying all functionalities in the native Windows Command Prompt (cmd.exe) or PowerShell for optimal interoperability. Otherwise, problems can occur when building the mtxs extension on Windows, locally, or in the cloud.

# The CAP Cookbook Guides and Recipes for Common Tasks { .subtitle} The following figure illustrates a walkthrough of the most prominent tasks within CAP's universe of discourse (aka scope). The guides contained in this section provide details and instructions about each. ![The graphic groups topics into three phases: Development, Deploy, Use. The development phase covers topics like domain modeling, sing and providing services, databases and frontends. The deploy phase covers the deployment as well as CI/CD, monitoring and publishing APIs and packages for reuse. The use phase is about the subscription flow of multitenant applications and about customizing and extending those applications. ](assets/cookbook-overview.drawio.svg)
# Providing Services This guide introduces how to define and implement services, leveraging generic implementations provided by the CAP runtimes, complemented by domain-specific custom logic. ## Intro: Core Concepts {#introduction} The following sections give a brief overview of CAP's core concepts. ### Service-Centric Paradigm A CAP application commonly provides services defined in CDS models and served by the CAP runtimes. Every active thing in CAP is a service. They embody the behavioral aspects of a domain in terms of exposed entities, actions, and events. ![This graphic is explained in the accompanying text.](assets/providing-services/service-centric-paradigm.drawio.svg) ### Ubiquitous Events At runtime, everything happening is in response to events. CAP features a ubiquitous notion of events, which represent both, *requests* coming in through **synchronous** APIs, as well as **asynchronous** *event messages*, blurring the line between both worlds. ![This graphic shows that consumers send events to services and that there are hooks, so that event handlers can react on those events.](assets/providing-services/services-events.drawio.svg) ### Event Handlers Service providers basically react on events in event handlers, plugged in to respective hooks provided by the core service runtimes. ## Service Definitions ### Services as APIs In its most basic form, a service definition simply declares the data entities and operations it serves. For example: ```cds service BookshopService { entity Books { key ID : UUID; title : String; author : Association to Authors; } entity Authors { key ID : UUID; name : String; books : Association to many Books on books.author = $self; } action submitOrder (book : Books:ID, quantity : Integer); } ``` This definition effectively defines the API served by `BookshopService`. ![This graphic is explained in the accompanying text.](assets/providing-services/service-apis.drawio.svg) Simple service definitions like that are all we need to run full-fledged servers out of the box, served by CAP's generic runtimes, without any implementation coding required. ### Services as Facades In contrast to the all-in-one definition above, services usually expose views, aka projections, on underlying domain model entities: ```cds using { sap.capire.bookshop as my } from '../db/schema'; service BookshopService { entity Books as projection on my.Books; entity Authors as projection on my.Authors; action submitOrder (book : Books:ID, quantity : Integer); } ``` This way, services become facades to encapsulated domain data, exposing different aspects tailored to respective use cases. ![This graphic is explained in the accompanying text.](assets/providing-services/service-as-facades.drawio.svg) ### Denormalized Views Instead of exposing access to underlying data in a 1:1 fashion, services frequently expose denormalized views, tailored to specific use cases. For example, the following service definition, undiscloses information about maintainers from end users and also [marks the entities as `@readonly`](#readonly): ```cds using { sap.capire.bookshop as my } from '../db/schema'; /** For serving end users */ service CatalogService @(path:'/browse') { /** For displaying lists of Books */ @readonly entity ListOfBooks as projection on Books excluding { descr }; /** For display in details pages */ @readonly entity Books as projection on my.Books { *, author.name as author } excluding { createdBy, modifiedBy }; } ``` [Learn more about **CQL** the language used for `projections`.](../cds/cql){.learn-more} [See also: Prefer Single-Purposed Services!](#single-purposed-services){.learn-more} [Find above sources in **cap/samples**.](https://github.com/sap-samples/cloud-cap-samples/tree/main/bookshop/srv/cat-service.cds){ .learn-more} ### Auto-Exposed Entities Annotate entities with `@cds.autoexpose` to automatically include them in services containing entities with Association referencing to them. For example, this is commonly done for code list entities in order to serve Value Lists dropdowns on UIs: ```cds service Zoo { entity Foo { //... code : Association to SomeCodeList; } } @cds.autoexpose entity SomeCodeList {...} ``` [Learn more about Auto-Exposed Entities in the CDS reference docs.](../cds/cdl#auto-expose){.learn-more} ### Redirected Associations When exposing related entities, associations are automatically redirected. This ensures that clients can navigate between projected entities as expected. For example: ```cds service AdminService { entity Books as projection on my.Books; entity Authors as projection on my.Authors; //> AdminService.Authors.books refers to AdminService.Books } ``` [Learn more about Redirected Associations in the CDS reference docs.](../cds/cdl#auto-redirect){.learn-more} ## Generic Providers The CAP runtimes for [Node.js](../node.js/) and [Java](../java/) provide a wealth of generic implementations, which serve most requests automatically, with out-of-the-box solutions to recurring tasks such as search, pagination, or input validation — the majority of this guide focuses on these generic features. In effect, a service definition [as introduced above](#service-definitions) is all we need to run a full-fledged server out of the box. The need for coding reduces to real custom logic specific to a project's domain → section [Custom Logic](#custom-logic) picks that up. ### Serving CRUD Requests {#serving-crud} The CAP runtimes for [Node.js](../node.js/) and [Java](../java/) provide generic handlers, which automatically serve all CRUD requests to entities for CDS-modelled services on top of a default [primary database](databases). This comprises read and write operations like that: * `GET /Books/201` → reading single data entities * `GET /Books?...` → reading data entity sets with advanced query options * `POST /Books {....}` → creating new data entities * `PUT/PATCH /Books/201 {...}` → updating data entities * `DELETE /Books/201` → deleting data entities
::: warning No filtering and sorting for virtual elements CAP runtimes delegate filtering and sorting to the database. Therefore filtering and sorting is not available for `virtual` elements. ::: ### Deep Reads and Writes CDS and the runtimes have advanced support for modeling and serving document-oriented data. The runtimes provide generic handlers for serving deeply nested document structures out of the box as documented in here. #### Deep `READ` You can read deeply nested documents by *expanding* along associations or compositions. For example, like this in OData: :::code-group ```http GET .../Orders?$expand=header($expand=items) ``` ```js[cds.ql] SELECT.from ('Orders', o => { o.ID, o.title, o.header (h => { h.ID, h.status, h.items('*') }) }) ``` [Learn more about `cds.ql`](../node.js/cds-ql){.learn-more} ::: Both would return an array of nested structures as follows: ```js [{ ID:1, title: 'first order', header: { // to-one ID:2, status: 'open', items: [{ // to-many ID:3, description: 'first order item' },{ ID:4, description: 'second order item' }] } }, ... ] ``` #### Deep `INSERT` Create a parent entity along with child entities in a single operation, for example, like that: :::code-group ```http POST .../Orders { ID:1, title: 'new order', header: { // to-one ID:2, status: 'open', items: [{ // to-many ID:3, description: 'child of child entity' },{ ID:4, description: 'another child of child entity' }] } } ``` ::: Note that Associations and Compositions are handled differently in (deep) inserts and updates: - Compositions → runtime **deeply creates or updates** entries in target entities - Associations → runtime **fills in foreign keys** to *existing* target entries For example, the following request would create a new `Book` with a *reference* to an existing `Author`, with `{ID:12}` being the foreign key value filled in for association `author`: ```http POST .../Books { ID:121, title: 'Jane Eyre', author: {ID:12} } ``` #### Deep `UPDATE` Deep `UPDATE` of the deeply nested documents look very similar to deep `INSERT`: :::code-group ```http PUT .../Orders/1 { title: 'changed title of existing order', header: { ID:2, items: [{ ID:3, description: 'modified child of child entity' },{ ID:5, description: 'new child of child entity' }] }] } ``` ::: Depending on existing data, child entities will be created, updated, or deleted as follows: - entries existing on the database, but not in the payload, are deleted → for example, `ID:4` - entries existing on the database, and in the payload are updated → for example, `ID:3` - entries not existing on the database are created → for example, `ID:5` **`PUT` vs `PATCH`** — Omitted fields get reset to `default` values or `null` in case of `PUT` requests; they are left untouched for `PATCH` requests. Omitted compositions have no effect, whether during `PATCH` or during `PUT`. That is, to delete all children, the payload must specify `null` or `[]`, respectively, for the to-one or to-many composition. #### Deep `DELETE` Deleting a root of a composition hierarchy results in a cascaded delete of all nested children. :::code-group ```sql DELETE .../Orders/1 -- would also delete all headers and items ``` ::: ### Auto-Generated Keys On `CREATE` operations, `key` elements of type `UUID` are filled in automatically. In addition, on deep inserts and upserts, respective foreign keys of newly created nested objects are filled in accordingly. For example, given a model like that: ```cds entity Orders { key ID : UUID; title : String; Items : Composition of many OrderItems on Items.order = $self; } entity OrderItems { key order : Association to Orders; key pos : Integer; descr: String; } ``` When creating a new `Order` with nested `OrderItems` like that: ```js POST .../Orders { title: 'Order #1', Items: [ { pos:1, descr: 'Item #1' }, { pos:2, descr: 'Item #2' } ] } ``` CAP runtimes will automatically fill in `Orders.ID` with a new uuid, as well as the nested `OrderItems.order.ID` referring to the parent. ### Searching Data CAP runtimes provide out-of-the-box support for advanced search of a given text in all textual elements of an entity including nested entities along composition hierarchies. A typical search request looks like that: ```js GET .../Books?$search=Heights ``` That would basically search for occurrences of `"Heights"` in all text fields of Books, that is, in `title` and `descr` using database-specific `contains` operations (for example, using `like '%Heights%'` in standard SQL). #### The `@cds.search` Annotation {#cds-search} By default search is limited to the elements of type `String` of an entity that aren't [calculated](../cds/cdl#calculated-elements) or [virtual](../cds/cdl#virtual-elements). Yet, sometimes you may want to deviate from this default and specify a different set of searchable elements, or to extend the search to associated entities. Use the `@cds.search` annotation to do so. The general usage is: ```cds @cds.search: { element1, // included element2 : true, // included element3 : false, // excluded assoc1, // extend to searchable elements in target entity assoc2.elementA // extend to a specific element in target entity } entity E { } ``` [Learn more about the syntax of annotations.](../cds/cdl#annotations){.learn-more} #### Including Fields ```cds @cds.search: { title } entity Books { ... } ``` Searches the `title` element only. ##### Extend Search to *Associated* Entities ::: warning Node.js: Only w/ streamlined database services For Node.js projects, this feature is only available with the [streamlined `@cap-js/` database services](../releases/archive/2024/jun24#new-database-services-ga) (default with `@sap/cds` >= 8) ::: ```cds @cds.search: { author } entity Books { ... } @cds.search: { biography: false } entity Authors { ... } ``` Searches all elements of the `Books` entity, as well as all searchable elements of the associated `Authors` entity. Which elements of the associated entity are searchable is determined by the `@cds.search` annotation on the associated entity. So, from `Authors`, all elements of type `String` are searched but `biography` is excluded. ##### Extend to Individual Elements in Associated Entities ```cds @cds.search: { author.name } entity Books { ... } ``` Searches only in the element `name` of the associated `Authors` entity. #### Excluding Fields ```cds @cds.search: { isbn: false } entity Books { ... } ``` Searches all elements of type `String` excluding the element `isbn`, which leaves the `title` and `descr` elements to be searched. ::: tip You can explicitly annotate calculated elements to make them searchable, even though they aren't searchable by default. The virtual elements won't be searchable even if they're explicitly annotated. ::: #### Fuzzy Search on SAP HANA Cloud {#fuzzy-search} > Prerequisite: For CAP Java, you need to run in [`HEX` optimization mode](../java/cqn-services/persistence-services#sql-optimization-mode) on SAP HANA Cloud and enable cds.sql.hana.search.fuzzy = true Fuzzy search is a fault-tolerant search feature of SAP HANA Cloud, which returns records even if the search term contains additional characters, is missing characters, or has typographical errors. You can configure the fuzziness in the range [0.0, 1.0]. The value 1.0 enforces exact search. - Java: cds.sql.hana.search.fuzzinessThreshold = 0.8 - Node.js:cds.hana.fuzzy = 0.7 Override the fuzziness for elements, using the `@Search.fuzzinessThreshold` annotation: ```cds entity Books { @Search.fuzzinessThreshold: 0.7 title : String; } ``` The relevance of a search match depends on the weight of the element causing the match. By default, all [searchable elements](#cds-search) have equal weight. To adjust the weight of an element, use the `@Search.ranking` annotation. Allowed values are HIGH, MEDIUM (default), and LOW: ```cds entity Books { @Search.ranking: HIGH title : String; @Search.ranking: LOW publisherName : String; } ``` ::: tip Wildcards in search terms When using wildcards in search terms, an *exact pattern search* is performed. Supported wildcards are '*' matching zero or more characters and '?' matching a single character. You can escape wildcards using '\\'. ::: ### Pagination & Sorting #### Implicit Pagination By default, the generic handlers for READ requests automatically **truncate** result sets to a size of 1,000 records max. If there are more entries available, a link is added to the response allowing clients to fetch the next page of records. The OData response body for truncated result sets contains a `nextLink` as follows: ```http GET .../Books >{ value: [ {... first record ...}, {... second record ...}, ... ], @odata.nextLink: "Books?$skiptoken=1000" } ``` To retrieve the next page of records from the server, the client would use this `nextLink` in a follow-up request, like so: ```http GET .../Books?$skiptoken=1000 ``` On firing this query, you get the second set of 1,000 records with a link to the next page, and so on, until the last page is returned, with the response not containing a `nextLink`. ::: warning Per OData specification for [Server Side Paging](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_ServerDrivenPaging), the value of the `nextLink` returned by the server must not be interpreted or changed by the clients. ::: #### Reliable Pagination > Note: This feature is available only for OData V4 endpoints. Using a numeric skip token based on the values of `$skip` and `$top` can result in duplicate or missing rows if the entity set is modified between the calls. _Reliable Pagination_ avoids this inconsistency by generating a skip token based on the values of the last row of a page. The reliable pagination is available with following limitations: - Results of functions or arithmetic expressions can't be used in the `$orderby` option (explicit ordering). - The elements used in the `$orderby` of the request must be of simple type. - All elements used in `$orderby` must also be included in the `$select` option, if it's set. - Complex [concatenations](../advanced/odata#concat) of result sets aren't supported. ::: warning Don't use reliable pagination if an entity set is sorted by elements that contain sensitive information, the skip token could reveal the values of these elements. ::: The feature can be enabled with the following [configuration options](../node.js/cds-env#project-settings) set to `true`: - Java: cds.query.limit.reliablePaging.enabled: true - Node.js: cds.query.limit.reliablePaging: true #### Paging Limits You can configure default and maximum page size limits in your [project configuration](../node.js/cds-env#project-settings) as follows: ```json "cds": { "query": { "limit": { "default": 20, //> no default "max": 100 //> default 1000 } } } ``` - The **maximum limit** defines the maximum number of items that can get retrieved, regardless of `$top`. - The **default limit** defines the number of items that are retrieved if no `$top` was specified. ##### Annotation `@cds.query.limit` {#annotation-cds-query-limit} You can override the defaults by applying the `@cds.query.limit` annotation on the service or entity level, as follows: ```cds @cds.query.limit: { default?, max? } | Number ``` The limit definitions for `CatalogService` and `AdminService` in the following example are equivalent. ```cds @cds.query.limit.default: 20 @cds.query.limit.max: 100 service CatalogService { // ... } @cds.query.limit: { default: 20, max: 100 } service AdminService { // ... } ``` `@cds.query.limit` can be used as shorthand if no default limit needs to be specified at the same level. ```cds @cds.query.limit: 100 service CatalogService { entity Books as projection on my.Books; //> pages at 100 @cds.query.limit: 20 entity Authors as projection on my.Authors; //> pages at 20 } service AdminService { entity Books as projection on my.Books; //> pages at 1000 (default) } ``` ##### Precedence The closest limit applies, that means, an entity-level limit overrides that of its service, and a service-level limit overrides the global setting. The value `0` disables the respective limit at the respective level. ```cds @cds.query.limit.default: 20 service CatalogService { @cds.query.limit.max: 100 entity Books as projection on my.Books; //> default = 20 (from CatalogService), max = 100 @cds.query.limit: 0 entity Authors as projection on my.Authors; //> no default, max = 1,000 (from environment) } ``` #### Implicit Sorting Paging requires implied sorting, otherwise records might be skipped accidentally when reading follow-up pages. By default the entity's primary key is used as a sort criterion. For example, given a service definition like this: ```cds service CatalogService { entity Books as projection on my.Books; } ``` The SQL query executed in response to incoming requests to Books will be enhanced with an additional order-by clause as follows: ```sql SELECT ... from my_Books ORDER BY ID; -- default: order by the entity's primary key ``` If the request specifies a sort order, for example, `GET .../Books?$orderby=author`, both are applied as follows: ```sql SELECT ... from my_Books ORDER BY author, -- request-specific order has precedence ID; -- default order still applied in addition ``` We can also define a default order when serving books as follows: ```cds service CatalogService { entity Books as projection on my.Books order by title asc; } ``` Now, the resulting order by clauses are as follows for `GET .../Books`: ```sql SELECT ... from my_Books ORDER BY title asc, -- from entity definition ID; -- default order still applied in addition ``` ... and for `GET .../Books?$orderby=author`: ```sql SELECT ... from my_Books ORDER BY author, -- request-specific order has precedence title asc, -- from entity definition ID; -- default order still applied in addition ``` ### Concurrency Control CAP runtimes support different ways to avoid lost-update situations as documented in the following. Use _optimistic locking_ to _detect_ concurrent modification of data _across requests_. The implementation relies on [ETags](#etag). Use _pessimistic locking_ to _protect_ data from concurrent modification by concurrent _transactions_. CAP leverages database locks for [pessimistic locking](#select-for-update). #### Conflict Detection Using ETags {#etag} The CAP runtimes support optimistic concurrency control and caching techniques using ETags. An ETag identifies a specific version of a resource found at a URL. Enable ETags by adding the `@odata.etag` annotation to an element to be used to calculate an ETag value as follows: ```cds using { managed } from '@sap/cds/common'; entity Foo : managed {...} annotate Foo with { modifiedAt @odata.etag } ``` > The value of an ETag element should uniquely change with each update per row. > The `modifiedAt` element from the [pre-defined `managed` aspect](../cds/common#aspect-managed) is a good candidate, as this is automatically updated. > You could also use update counters or UUIDs, which are recalculated on each update. You use ETags when updating, deleting, or invoking the action bound to an entity by using the ETag value in an `If-Match` or `If-None-Match` header. The following examples represent typical requests and responses: ```http POST Employees { ID:111, name:'Name' } > 201 Created {'@odata.etag': 'W/"2000-01-01T01:10:10.100Z"',...} //> Got new ETag to be used for subsequent requests... ``` ```http GET Employees/111 If-None-Match: "2000-01-01T01:10:10.100Z" > 304 Not Modified // Record was not changed ``` ```http GET Employees/111 If-Match: "2000-01-01T01:10:10.100Z" > 412 Precondition Failed // Record was changed by another user ``` ```http UPDATE Employees/111 If-Match: "2000-01-01T01:10:10.100Z" > 200 Ok {'@odata.etag': 'W/"2000-02-02T02:20:20.200Z"',...} //> Got new ETag to be used for subsequent requests... ``` ```http UPDATE Employees/111 If-Match: "2000-02-02T02:20:20.200Z" > 412 Precondition Failed // Record was modified by another user ``` ```http DELETE Employees/111 If-Match: "2000-02-02T02:20:20.200Z" > 412 Precondition Failed // Record was modified by another user ``` If the ETag validation detects a conflict, the request typically needs to be retried by the client. Hence, optimistic concurrency should be used if conflicts occur rarely. #### Pessimistic Locking {#select-for-update} _Pessimistic locking_ allows you to lock the selected records so that other transactions are blocked from changing the records in any way. Use _exclusive_ locks when reading entity data with the _intention to update_ it in the same transaction and you want to prevent the data to be read or updated in a concurrent transaction. Use _shared_ locks if you only need to prevent the entity data to be updated in a concurrent transaction, but don't want to block concurrent read operations. The records are locked until the end of the transaction by commit or rollback statement. [Learn more about using the `SELECT ... FOR UPDATE` statement in the Node.js runtime.](../node.js/cds-ql#forupdate){.learn-more} [Learn more about using the `Select.lock()` method in the Java runtime.](../java/working-with-cql/query-api#write-lock){.learn-more} ::: warning Pessimistic locking is not supported by SQLite. H2 supports exclusive locks only. ::: ## Input Validation CAP runtimes automatically validate user input, controlled by the following annotations. ### `@readonly` Elements annotated with `@readonly`, as well as [_calculated elements_](../cds/cdl#calculated-elements), are protected against write operations. That is, if a CREATE or UPDATE operation specifies values for such fields, these values are **silently ignored**. By default [`virtual` elements](../cds/cdl#virtual-elements) are also _calculated_. ::: tip The same applies for fields with the [OData Annotations](../advanced/odata#annotations) `@FieldControl.ReadOnly` (static), `@Core.Computed`, or `@Core.Immutable` (the latter only on UPDATEs). ::: ### `@mandatory` Elements marked with `@mandatory` are checked for nonempty input: `null` and (trimmed) empty strings are rejected. ```cds service Sue { entity Books { key ID : UUID; title : String @mandatory; } } ``` In addition to server-side input validation as introduced above, this adds a corresponding `@FieldControl` annotation to the EDMX so that OData / Fiori clients would enforce a valid entry, thereby avoiding unnecessary request roundtrips: ```xml ``` ### `@Common.FieldControl` {#common-fieldcontrol} The input validation for `@Common.FieldControl: #Mandatory` and `@Common.FieldControl: #ReadOnly` is done from the CAP runtimes automatically. ::: warning Custom validations are required when using static or dynamic numeric values, for example, `@Common.FieldControl: 1` or `@Common.FieldControl: integer_field`. ::: ### `@assert .unique` Annotate an entity with `@assert.unique.`, specifying one or more element combinations to enforce uniqueness checks on all CREATE and UPDATE operations. For example: ```cds @assert.unique: { locale: [ parent, locale ], timeslice: [ parent, validFrom ], } entity LocalizedTemporalData { key record_ID : UUID; // technical primary key parent : Association to Data; locale : String; validFrom : Date; validTo : Date; } ``` {.indent} This annotation is applicable to entities, which result in tables in SQL databases only. The value of the annotation is an array of paths referring to elements in the entity. These elements may be of a scalar type, structs, or managed associations. Individual foreign keys or unmanaged associations are not supported. If structured elements are specified, the unique constraint will contain all columns stemming from it. If the path points to a managed association, the unique constraint will contain all foreign key columns stemming from it. ::: tip You don't need to specify `@assert.unique` constraints for the primary key elements of an entity as these are automatically secured by a SQL `PRIMARY KEY` constraint. ::: ### `@assert .target` Annotate a [managed to-one association](../cds/cdl#managed-associations) of a CDS model entity definition with the `@assert.target` annotation to check whether the target entity referenced by the association (the reference's target) exists. In other words, use this annotation to check whether a non-null foreign key input in a table has a corresponding primary key in the associated/referenced target table. You can check whether multiple targets exist in the same transaction. For example, in the `Books` entity, you could annotate one or more managed to-one associations with the `@assert.target` annotation. However, it is assumed that dependent values were inserted before the current transaction. For example, in a deep create scenario, when creating a book, checking whether an associated author exists that was created as part of the same deep create transaction isn't supported, in this case, you will get an error. The `@assert.target` check constraint is meant to **validate user input** and not to ensure referential integrity. Therefore only `CREATE`, and `UPDATE` events are supported (`DELETE` events are not supported). To ensure that every non-null foreign key in a table has a corresponding primary key in the associated/referenced target table (ensure referential integrity), the [`@assert.integrity`](databases#database-constraints) constraint must be used instead. If the reference's target doesn't exist, an HTTP response (error message) is provided to HTTP client applications and logged to stdout in debug mode. The HTTP response body's content adheres to the standard OData specification for an error [response body](https://docs.oasis-open.org/odata/odata-json-format/v4.01/cs01/odata-json-format-v4.01-cs01.html#sec_ErrorResponse). #### Example Add `@assert.target` annotation to the service definition as previously mentioned: ```cds entity Books { key ID : UUID; title : String; author : Association to Authors @assert.target; } entity Authors { key ID : UUID; name : String; books : Association to many Books on books.author = $self; } ``` **HTTP Request** — *assume that an author with the ID `"796e274a-c3de-4584-9de2-3ffd7d42d646"` doesn't exist in the database* ```http POST Books HTTP/1.1 Accept: application/json;odata.metadata=minimal Prefer: return=minimal Content-Type: application/json;charset=UTF-8 {"author_ID": "796e274a-c3de-4584-9de2-3ffd7d42d646"} ``` **HTTP Response** ```http HTTP/1.1 400 Bad Request odata-version: 4.0 content-type: application/json;odata.metadata=minimal {"error": { "@Common.numericSeverity": 4, "code": "400", "message": "Value doesn't exist", "target": "author_ID" }} ``` ::: tip In contrast to the `@assert.integrity` constraint, whose check is performed on the underlying database layer, the `@assert.target` check constraint is performed on the application service layer before the custom application handlers are called. ::: ::: warning Cross-service checks are not supported. It is expected that the associated entities are defined in the same service. ::: ::: warning The `@assert.target` check constraint relies on database locks to ensure accurate results in concurrent scenarios. However, locking is a database-specific feature, and some databases don't permit to lock certain kinds of objects. On SAP HANA, for example, views with joins or unions can't be locked. Do not use `@assert.target` on such artifacts/entities. ::: ### `@assert .format` Allows you to specify a regular expression string (in ECMA 262 format in CAP Node.js and java.util.regex.Pattern format in CAP Java) that all string input must match. ```cds entity Foo { bar : String @assert.format: '[a-z]ear'; } ``` ### `@assert .range` Allows you to specify `[ min, max ]` ranges for elements with ordinal types — that is, numeric or date/time types. For `enum` elements, `true` can be specified to restrict all input to the defined enum values. ```cds entity Foo { bar : Integer @assert.range: [ 0, 3 ]; boo : Decimal @assert.range: [ 2.1, 10.25 ]; car : DateTime @assert.range: ['2018-10-31', '2019-01-15']; zoo : String @assert.range enum { high; medium; low; }; } ``` #### ... with open intervals By default, specified `[min,max]` ranges are interpreted as closed intervals, that means, the performed checks are `min ≤ input ≤ max`. You can also specify open intervals by wrapping the *min* and/or *max* values into parenthesis like that: ```cds @assert.range: [(0),100] // 0 < input ≤ 100 @assert.range: [0,(100)] // 0 ≤ input < 100 @assert.range: [(0),(100)] // 0 < input < 100 ``` In addition, you can use an underscore `_` to represent *Infinity* like that: ```cds @assert.range: [(0),_] // positive numbers only, _ means +Infinity here @assert.range: [_,(0)] // negative number only, _ means -Infinity here ``` > Basically values wrapped in parentheses _`(x)`_ can be read as _excluding `x`_ for *min* or *max*. Note that the underscore `_` doesn't have to be wrapped into parenthesis, as by definition no number can be equal to *Infinity* . ::: warning Support in latest runtimes Support for open intervals and infinity has been added to CAP Node.js, i.e. `@sap/cds` version **8.5**. Support in CAP Java is **not yet available** but will follow soon. ::: ### `@assert .notNull` Annotate a property with `@assert.notNull: false` to have it ignored during the generic not null check, for example if your persistence fills it automatically. ```cds entity Foo { bar : String not null @assert.notNull: false; } ``` ## Custom Logic As most standard tasks and use cases are covered by [generic service providers](#generic-providers), the need to add service implementation code is greatly reduced and minified, and hence the quantity of individual boilerplate coding. The remaining cases that need custom handlers, reduce to real custom logic, specific to your domain and application, such as: - Domain-specific programmatic [Validations](#input-validation) - Augmenting result sets, for example to add computed fields for frontends - Programmatic [Authorization Enforcements](/guides/security/authorization#enforcement) - Triggering follow-up actions, for example calling other services or emitting outbound events in response to inbound events - And more... In general, all the things not (yet) covered by generic handlers **In Node.js**, the easiest way to add custom implementations for services is through equally named _.js_ files placed next to a service definition's _.cds_ file: ```sh ./srv - cat-service.cds # service definitions - cat-service.js # service implementation ... ``` [Learn more about providing service implementations in Node.js.](../node.js/core-services#implementing-services){.learn-more} **In Java**, you'd assign `EventHandler` classes using dependency injection as follows: ```Java @Component @ServiceName("org.acme.Foo") public class FooServiceImpl implements EventHandler {...} ``` [Learn more about Event Handler classes in Java.](../java/event-handlers/#handlerclasses){.learn-more} ### Custom Event Handlers Within your custom implementations, you can register event handlers like that: ::: code-group ```js [Node.js] module.exports = function (){ this.on ('submitOrder', (req)=>{...}) //> custom actions this.on ('CREATE',`Books`, (req)=>{...}) this.before ('UPDATE',`*`, (req)=>{...}) this.after ('READ',`Books`, (books)=>{...}) } ``` ```Java @Component @ServiceName("BookshopService") public class BookshopServiceImpl implements EventHandler { @On(event="submitOrder") public void onSubmitOrder (EventContext req) {...} @On(event="CREATE", entity="Books") public void onCreateBooks (EventContext req) {...} @Before(event="UPDATE", entity="*") public void onUpdate (EventContext req) {...} @After(event="READ", entity="Books") public void onReadBooks (EventContext req) {...} } ``` ::: [Learn more about **adding event handlers in Node.js**.](../node.js/core-services#srv-on-before-after){.learn-more} [Learn more about **adding event handlers in Java**.](../java/event-handlers/#handlerclasses){.learn-more} ### Hooks: `on`, `before`, `after` In essence, event handlers are functions/method registered to be called when a certain event occurs, with the event being a custom operation, like `submitOrder`, or a CRUD operation on a certain entity, like `READ Books`; in general following this scheme: - `` , `` , `[]` → handler function CAP allows to plug in event handlers to these different hooks, that is phases during processing a certain event: - `on` handlers run _instead of_ the generic/default handlers. - `before` handlers run _before_ the `on` handlers - `after` handlers run _after_ the `on` handlers, and get the result set as input `on` handlers form an *interceptor* stack: the topmost handler getting called by the framework. The implementation of this handler is in control whether to delegate to default handlers down the stack or not. `before` and `after` handlers are *listeners*: all registered listeners are invoked in parallel. If one vetoes / throws an error the request fails. ### Within Event Handlers {#handler-impls} Event handlers all get a uniform _Request_/_Event Message_ context object as their primary argument, which, among others, provides access to the following information: - The `event` name — that is, a CRUD method name, or a custom-defined one - The `target` entity, if any - The `query` in [CQN](../cds/cqn) format, for CRUD requests - The `data` payload - The `user`, if identified/authenticated - The `tenant` using your SaaS application, if enabled [Learn more about **implementing event handlers in Node.js**.](../node.js/events#cds-request){.learn-more} [Learn more about **implementing event handlers in Java**.](../java/event-handlers/#eventcontext){.learn-more} ## Actions & Functions In addition to common CRUD operations, you can declare domain-specific custom operations as shown below. These custom operations always need custom implementations in corresponding events handlers. You can define actions and functions in CDS models like that: ```cds service Sue { // unbound actions & functions function sum (x:Integer, y:Integer) returns Integer; function stock (id : Foo:ID) returns Integer; action add (x:Integer, to: Integer) returns Integer; // bound actions & functions entity Foo { key ID:Integer } actions { function getStock() returns Integer; action order (x:Integer) returns Integer; //bound to the collection and not a specific instance of Foo action customCreate (in: many $self, x: String) returns Foo; } } ``` [Learn more about modeling actions and functions in CDS.](../cds/cdl#actions){.learn-more} The differentiation between *Actions* and *Functions* as well as *bound* and *unbound* stems from the OData specifications, and in essence is as follows: - **Actions** modify data in the server - **Functions** retrieve data - **Unbound** actions/functions are like plain unbound functions in JavaScript. - **Bound** actions/functions always receive the bound entity's primary key as implicit first argument, similar to `this` pointers in Java or JavaScript. The exception are bound actions to collections, which are bound against the collection and not a specific instance of the entity. An example use case are custom create actions for the SAP Fiori elements UI. ### Implementing Actions / Functions In general, implement actions or functions like that: ```js module.exports = function Sue(){ this.on('sum', ({data:{x,y}}) => x+y) this.on('add', ({data:{x,to}}) => stocks[to] += x) this.on('stock', ({data:{id}}) => stocks[id]) this.on('getStock','Foo', ({params:[id]}) => stocks[id]) this.on('order','Foo', ({params:[id],data:{x}}) => stocks[id] -= x) } ``` Event handlers for actions or functions are very similar to those for CRUD events, with the name of the action/function replacing the name of the CRUD operations. No entity is specific for unbound actions/functions. **Method-style Implementations** in Node.js, you can alternatively implement actions and functions using conventional JavaScript methods with subclasses of `cds.Service`: ```js module.exports = class Sue extends cds.Service { sum(x,y) { return x+y } add(x,to) { return stocks[to] += x } stock(id) { return stocks[id] } getStock(Foo,id) { return stocks[id] } order(Foo,id,x) { return stocks[id] -= x } } ``` ### Calling Actions / Functions **HTTP Requests** to call the actions/function declared above look like that: ```js GET .../sue/sum(x=1,y=2) // unbound function GET .../sue/stock(id=2) // unbound function POST .../sue/add {"x":1,"to":2} // unbound action GET .../sue/Foo(2)/Sue.getStock() // bound function POST .../sue/Foo(2)/Sue.order {"x":1} // bound action ``` > Note: You always need to add the `()` for functions, even if no arguments are required. The OData standard specifies that bound actions/functions need to be prefixed with the service's name. In the previous example, entity `Foo` has a bound action `order`. That action must be called via `/Foo(2)/Sue.order` instead of simply `/Foo(2)/order`. > For convenience, the CAP Node.js runtime also allows the following: > - Call bound actions/functions without prefixing them with the service name. > - Omit the `()` if no parameter is required. > - Use query options to provide function parameters like `sue/sum?x=1&y=2`
**Programmatic** usage via **generic APIs** would look like this for Node.js: ```js const srv = await cds.connect.to('Sue') // unbound actions/functions await srv.send('sum',{x:1,y:2}) await srv.send('add',{x:11,to:2}) await srv.send('stock',{id:2}) // bound actions/functions await srv.send('getStock','Foo',{id:2}) //for passing the params property, use this syntax await srv.send({ event: 'order', entity: 'Foo', data: {x:3}, params: [2]}) ``` > Note: Always pass the target entity name as second argument for bound actions/functions.
**Programmatic** usage via **typed API** — Node.js automatically equips generated service instances with specific methods matching the definitions of actions/functions found in the services' model. This allows convenient usage like that: ```js const srv = await cds.connect.to(Sue) // unbound actions/functions srv.sum(1,2) srv.add(11,2) srv.stock(2) // bound actions/functions srv.getStock('Foo',2) srv.order('Foo',2,3) ``` > Note: Even with that typed APIs, always pass the target entity name as second argument for bound actions/functions.

## Serving Media Data CAP provides out-of-the-box support for serving media and other binary data. ### Annotating Media Elements You can use the following annotations in the service model to indicate that an element in an entity contains media data. `@Core.MediaType` : Indicates that the element contains media data (directly or using a redirect). The value of this annotation is either a string with the contained MIME type (as shown in the first example), or is a path to the element that contains the MIME type (as shown in the second example). `@Core.IsMediaType` : Indicates that the element contains a MIME type. The `@Core.MediaType` annotation of another element can reference this element. `@Core.IsURL @Core.MediaType` : Indicates that the element contains a URL pointing to the media data (redirect scenario). `@Core.ContentDisposition.Filename` : Indicates that the element is expected to be displayed as an attachment, that is downloaded and saved locally. The value of this annotation is a path to the element that contains the Filename (as shown in the fourth example ). `@Core.ContentDisposition.Type` : Can be used to instruct the browser to display the element inline, even if `@Core.ContentDisposition.Filename` is specified, by setting to `inline` (see the fifth example). If omitted, the behavior is `@Core.ContentDisposition.Type: 'attachment'`. [Learn more how to enable stream support in SAP Fiori elements.](https://ui5.sap.com/#/topic/b236d32d48b74304887b3dd5163548c1){.learn-more} The following examples show these annotations in action: 1. Media data is stored in a database with a fixed media type `image/png`: ```cds entity Books { //... image : LargeBinary @Core.MediaType: 'image/png'; } ``` 2. Media data is stored in a database with a _variable_ media type: ```cds entity Books { //... image : LargeBinary @Core.MediaType: imageType; imageType : String @Core.IsMediaType; } ``` 3. Media data is stored in an external repository: ```cds entity Books { //... imageUrl : String @Core.IsURL @Core.MediaType: imageType; imageType : String @Core.IsMediaType; } ``` 4. Content disposition data is stored in a database with a _variable_ disposition: ```cds entity Authors { //... image : LargeBinary @Core.MediaType: imageType @Core.ContentDisposition.Filename: fileName; fileName : String; } ``` 5. The image shall have the suggested file name but be displayed inline nevertheless: ```cds entity Authors { //... image : LargeBinary @Core.MediaType: imageType @Core.ContentDisposition.Filename: fileName @Core.ContentDisposition.Type: 'inline'; fileName : String; } ``` [Learn more about the syntax of annotations.](../cds/cdl#annotations){.learn-more} ::: warning In case you rename the properties holding the media type or content disposition information in a projection, you need to update the annotation's value as well. ::: ### Reading Media Resources Read media data using `GET` requests of the form `/Entity()/mediaProperty`: ```cds GET ../Books(201)/image > Content-Type: application/octet-stream ``` > The response's `Content-Type` header is typically `application/octet-stream`. > Although allowed by [RFC 2231](https://datatracker.ietf.org/doc/html/rfc2231), Node.js does not support line breaks in HTTP headers. Hence, make sure you remove any line breaks from your `@Core.IsMediaType` content. Read media data with `@Core.ContentDisposition.Filename` in the model: ```cds GET ../Authors(201)/image > Content-Disposition: 'attachment; filename="foo.jpg"' ``` > The media data is streamed automatically. [Learn more about returning a custom streaming object (Node.js - beta).](../node.js/best-practices#custom-streaming-beta){.learn-more} ### Creating a Media Resource As a first step, create an entity without media data using a POST request to the entity. After creating the entity, you can insert a media property using the PUT method. The MIME type is passed in the `Content-Type` header. Here are some sample requests: ```cds POST ../Books Content-Type: application/json { } ``` ```cds PUT ../Books(201)/image Content-Type: image/png ``` > The media data is streamed automatically. ### Updating Media Resources The media data for an entity can be updated using the PUT method: ```cds PUT ../Books(201)/image Content-Type: image/png ``` > The media data is streamed automatically. ### Deleting Media Resources One option is to delete the complete entity, including all media data: ```cds DELETE ../Books(201) ``` Alternatively, you can delete a media data element individually: ```cds DELETE ../Books(201)/image ``` ### Using External Resources The following are requests and responses for the entity containing redirected media data from the third example, "Media data is stored in an external repository". > This format is used by OData-Version: 4.0. To be changed in OData-Version: 4.01. ```cds GET: ../Books(201) >{ ... image@odata.mediaReadLink: "http://other-server/image.jpeg", image@odata.mediaContentType: "image/jpeg", imageType: "image/jpeg" } ``` ### Conventions & Limitations #### General Conventions - Binary data in payloads must be a Base64 encoded string. - Binary data in URLs must have the format `binary''`. For example: ```http GET $filter=ID eq binary'Q0FQIE5vZGUuanM=' ``` #### Node.js Runtime Conventions and Limitations - The usage of binary data in some advanced constructs like the `$apply` query option and `/any()` might be limited. - On SQLite, binary strings are stored as plain strings, whereas a buffer is stored as binary data. As a result, if in a CDS query, a binary string is used to query data stored as binary, this wouldn't work. - Please note, that SQLite doesn't support streaming. That means, that LargeBinary fields are read as a whole (not in chunks) and stored in memory, which can impact performance. - SAP HANA Database Client for Node.js (HDB) and SAP HANA Client for Node.js (`@sap/hana-client`) packages handle binary data differently. For example, HDB automatically converts binary strings into binary data, whereas SAP HANA Client doesn't. - In the Node.js Runtime, all binary strings are converted into binary data according to SAP HANA property types. To disable this default behavior, you can set the environment variable cds.hana.base64_to_buffer: false. # Best Practices ## Single-Purposed Services {.best-practice} We strongly recommend designing your services for single use cases. Services in CAP are cheap, so there's no need to save on them. #### **DON'T:**{.bad} Single Services Exposing All Entities 1:1 The anti-pattern to that are single services exposing all underlying entities in your app in a 1:1 fashion. While that may save you some thoughts in the beginning, it's likely that it will result in lots of headaches in the long run: * They open huge entry doors to your clients with only few restrictions * Individual use-cases aren't reflected in your API design * You have to add numerous checks on a per-request basis... * Which have to reflect on the actual use cases in complex and expensive evaluations #### **DO:**{.good} One Service Per Use Case For example, let's assume that we have a domain model defining *Books* and *Authors* more or less as above, and then we add *Orders*. We could define the following services: ```cds using { my.domain as my } from './db/schema'; ``` ```cds /** Serves end users browsing books and place orders */ service CatalogService { @readonly entity Books as select from my.Books { ID, title, author.name as author }; @requires: 'authenticated-user' @insertonly entity Orders as projection on my.Orders; } ``` ```cds /** Serves registered users managing their account and their orders */ @requires: 'authenticated-user' service UsersService { @restrict: [{ grant: 'READ', where: 'buyer = $user' }] // limit to own ones @readonly entity Orders as projection on my.Orders; action cancelOrder ( ID:Orders.ID, reason:String ); } ``` ```cds /** Serves administrators managing everything */ @requires: 'authenticated-user' service AdminService { entity Books as projection on my.Books; entity Authors as projection on my.Authors; entity Orders as projection on my.Orders; } ``` These services serve different use cases and are tailored for each. Note, for example, that we intentionally don't expose the `Authors` entity to end users. ## Late-Cut Microservices {.best-practice} Compared to Microservices, CAP services are 'Nano'. As shown in the previous sections, you should design your application as a set of loosely coupled, single-purposed services, which can all be served embedded in a single-server process at first (that is, a monolith). Yet, given such loosely coupled services, and enabled by CAP's uniform way to define and consume services, you can decide later on to separate, deploy, and run your services as separate microservices, even without changing your models or code. This flexibility allows you to, again, focus on solving your domain problem first, and avoid the efforts and costs of premature microservice design and DevOps overhead, at least in the early phases of development. # Consuming Services ## Introduction If you want to use data from other services or you want to split your application into multiple microservices, you need a connection between those services. We call them **remote services**. As everything in CAP is a service, remote services are modeled the same way as internal services — using CDS. CAP supports service consumption with dedicated APIs to [import](#import-api) service definitions, [query](#execute-queries) remote services, [mash up](#building-mashups) services, and [work locally](#local-mocking) as much as possible. ### Feature Overview For outbound remote service consumption, the following features are supported: + OData V4 + OData V2 (Deprecated) + [Querying API](#querying-api-features) + [Projections on remote services](#supported-projection-features) ### Tutorials and Examples | Example | Description | | ---------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | [Capire Bookshop (Fiori)](https://github.com/sap-samples/cloud-cap-samples/tree/main/fiori) | Example, Node.js, CAP-to-CAP | | [Example Application (Node.js)](https://github.com/SAP-samples/cloud-cap-risk-management/tree/ext-service-s4hc-suppliers-ui) | Complete application from the end-to-end Tutorial | | [Example Application (Java)](https://github.com/SAP-samples/cloud-cap-risk-management/tree/ext-service-s4hc-suppliers-ui-java) | Complete application from the end-to-end Tutorial | ### Define Scenario Before you start your implementation, you should define your scenario. Answering the following questions gets you started: + What services (remote/CAP) are involved? + How do they interact? + What needs to be displayed on the UI? You have all your answers and know your scenario, go on reading about [external service APIs](#external-service-api), getting an API definition from [the SAP Business Accelerator Hub](#from-api-hub) or [from a CAP project](#from-cap-service), and [importing an API definition](#import-api) to your project. #### Sample Scenario from End-to-End Tutorial ![A graphic showing the flow for one possible scenario. A user can either view risks or view the suppliers. The suppliers master data is already available from a system and is consumed in an application that enables the user to add the risks. From the maintained risks the user can get information about the supplier connected to a risk. From the supplier view, it's also possible to get details about a risk that is associated with a supplier. The user can block/unblock suppliers from the risk view.](./assets/using-services/risk-mgmt.drawio.svg){} ::: info _User Story_ A company wants to ensure that goods are only sourced from suppliers with acceptable risks. There shall be a software system, that allows a clerk to maintain risks for suppliers and their mitigations. The system shall block the supplier used if risks can't be mitigated. ::: The application is an extension for SAP S/4HANA. It deals with _risks_ and _mitigations_ that are local entities in the application and _suppliers_ that are stored in SAP S/4HANA Cloud. The application helps to reduce risks associated with suppliers by automatically blocking suppliers with a high risk using a [remote API Call](#execute-queries). ##### Integrate The user picks a supplier from the list. That list is coming [from the remote system and is exposed by the CAP application](#expose-remote-services). Then the user does a risk assessment. Additional supplier data, like name and blocked status, should be displayed on the UI as well, by [integrating the remote supplier service into the local risk service](#integrate-remote-into-local-services). ##### Extend It should be also possible to search for suppliers and show the associated risks by extending the remote supplier service [with the local risk service](#extend-a-remote-by-a-local-service) and its risks. ## Get and Import an External Service API { #external-service-api } To communicate to remote services, CAP needs to know their definitions. Having the definitions in your project allows you to mock them during design time. These definitions are usually made available by the service provider. As they aren't defined within your application but imported from outside, they're called *external* service APIs in CAP. Service APIs can be provided in different formats. Currently, *EDMX* files for OData V2 and V4 are supported. ### From SAP Business Accelerator Hub { #from-api-hub} The [SAP Business Accelerator Hub](https://api.sap.com/) provides many relevant APIs from SAP. You can download API specifications in different formats. If available, use the EDMX format. The EDMX format describes OData interfaces. To download the [Business Partner API (A2X) from SAP S/4HANA Cloud](https://api.sap.com/api/API_BUSINESS_PARTNER/overview), go to section **API Resources**, select **API Specification**, and download the **EDMX** file. [Get more details in the end-to-end tutorial.](https://developers.sap.com/tutorials/btp-app-ext-service-add-consumption.html#07f89fdd-82b2-4987-aa86-070f1d836156){.learn-more} ### For a Remote CAP Service { #from-cap-service} We recommend using EDMX as exchange format. Export a service API to EDMX: ::: code-group ```sh [Mac/Linux] cds compile srv -s OrdersService -2 edmx > OrdersService.edmx ``` ```cmd [Windows] cds compile srv -s OrdersService -2 edmx > OrdersService.edmx ``` ```powershell [Powershell] cds compile srv -s OrdersService -2 edmx -o dest/ ``` ::: [You can try it with the orders sample in cap/samples.](https://github.com/SAP-samples/cloud-cap-samples/tree/master/orders){.learn-more} By default, CAP works with OData V4 and the EDMX export is in this protocol version as well. The `cds compile` command offers options for other OData versions and flavors, call `cds help compile` for more information. ::: warning Don't just copy the CDS file for a remote CAP service Simply copying CDS files from a different application comes with the following issues: - The effective service API depends on the used protocol. - CDS files often use includes, which can't be resolved anymore. - CAP creates unneeded database tables and views for all entities in the file. ::: ### Import API Definition { #import-api} Import the API to your project using `cds import`. ```sh cds import --as cds ``` > `` can be an EDMX (OData V2, OData V4), OpenAPI or AsyncAPI file. | Option | Description | | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--as cds` | The import creates a CDS file (for example _API_BUSINESS_PARTNER.cds_) instead of a CSN file. | This adds the API in CDS format to the _srv/external_ folder and also copies the input file into that folder.

Further, it adds the API as an external service to your _package.json_. You use this declaration later to connect to the remote service [using a destination](#use-destinations-with-node-js). ```json "cds": { "requires": { "API_BUSINESS_PARTNER": { "kind": "odata-v2", "model": "srv/external/API_BUSINESS_PARTNER" } } } ```

::: details Options and flags in _.cdsrc.json_ Alternatively, you can set the options and flags for `cds import` in your _.cdsrc.json_: ```json { "import": { "as": "cds", "force": true, "include_namespaces": "sap,c4c" } } ``` Now run `cds import ` - `--as` only supports these formats: "csn","cds", and "json" - `--force` is applicable only in combination with `--as` option. By default the `--force` flag is set to false. > If set to true, existing CSN/CDS files from previous imports are overwritten. ::: When importing the specification files, the `kind` is set according to the following mapping: | Imported Format | Used `kind` | |-----------------|--------------------------------| | OData V2 | `odata-v2` | | OData V4 | `odata` (alias for `odata-v4`) | | OpenAPI | `rest` | | AsyncAPI | `odata` | [Learn more about type mappings from OData to CDS and vice versa.](../tools/apis/cds-import#odata-type-mappings){.learn-more} ::: tip Always use OData V4 (`odata`) when calling another CAP service. ::: ::: warning Limitations Not all features of OData, OpenAPI, or AsyncAPI are supported in CAP which may lead to the rejection of the imported model by the CDS compiler or may result in a different API when rendered by CAP. Known limitations are cyclic type references and inheritance. :::

You need to configure remote services in Spring Boot's _application.yaml_: ::: code-group ```yaml [srv/src/main/resources/application.yaml] spring: config.activate.on-profile: cloud cds: remote.services: API_BUSINESS_PARTNER: type: "odata-v2" ``` ::: To work with remote services, add the following dependency to your Maven project: ```xml com.sap.cds cds-feature-remote-odata runtime ``` [Learn about all `cds.remote.services` configuration possibilities.](../java/developing-applications/properties#cds-remote-services){.learn-more}

## Local Mocking When developing your application, you can mock the remote service. ### Add Mock Data As for any other CAP service, you can add mocking data. The CSV file needs to be added to the _srv/external/data_ folder. {.node} The CSV file needs to be added to the _db/data_ folder. {.java} ::: code-group ```csv [API_BUSINESS_PARTNER-A_BusinessPartner.csv] BusinessPartner;BusinessPartnerFullName;BusinessPartnerIsBlocked 1004155;Williams Electric Drives;false 1004161;Smith Batteries Ltd;false 1004100;Johnson Automotive Supplies;true ``` ::: For Java, make sure to add the `--with-mocks` option to the `cds deploy` command used to generate the `schema.sql` in `srv/pom.xml`. This ensures that tables for the mocked remote entities are created in the database.{.java} [Find this source in the end-to-end tutorial](https://github.com/SAP-samples/cloud-cap-risk-management/blob/ext-service-s4hc-suppliers-ui-java/srv/external/data/API_BUSINESS_PARTNER-A_BusinessPartner.csv){.learn-more} [Get more details in the end-to-end tutorial.](https://developers.sap.com/tutorials/btp-app-ext-service-add-consumption.html#12ff20a2-e988-465f-a508-f527c7fc0c29){.learn-more} ### Run Local with Mocks Start your project with the imported service definition.

```sh cds watch ``` The service is automatically mocked, as you can see in the log output on server start. ```log{17} ... [cds] - model loaded from 8 file(s): ... ./srv/external/API_BUSINESS_PARTNER.cds ... [cds] - connect using bindings from: { registry: '~/.cds-services.json' } [cds] - connect to db > sqlite { database: ':memory:' } > filling sap.ui.riskmanagement.Mitigations from ./db/data/sap.ui.riskmanagement-Mitigations.csv > filling sap.ui.riskmanagement.Risks from ./db/data/sap.ui.riskmanagement-Risks.csv > filling API_BUSINESS_PARTNER.A_BusinessPartner from ./srv/external/data/API_BUSINESS_PARTNER-A_BusinessPartner.csv /> successfully deployed to sqlite in-memory db [cds] - serving RiskService { at: '/service/risk', impl: './srv/risk-service.js' } [cds] - mocking API_BUSINESS_PARTNER { at: '/api-business-partner' } // [!code focus] [cds] - launched in: 1.104s [cds] - server listening on { url: 'http://localhost:4004' } [ terminate with ^C ] ```

```sh mvn spring-boot:run ```

### Mock Associations You can't get data from associations of a mocked service out of the box. The associations of imported services lack information how to look up the associated records. This missing relation is expressed with an empty key definition at the end of the association declaration in the CDS model (`{ }`). ::: code-group ```cds{9} [srv/external/API_BUSINESS_PARTNER.cds] entity API_BUSINESS_PARTNER.A_BusinessPartner { key BusinessPartner : LargeString; BusinessPartnerFullName : LargeString; BusinessPartnerType : LargeString; ... to_BusinessPartnerAddress : Association to many API_BUSINESS_PARTNER.A_BusinessPartnerAddress { }; // [!code focus] }; entity API_BUSINESS_PARTNER.A_BusinessPartnerAddress { key BusinessPartner : String(10); key AddressID : String(10); ... }; ``` ::: To mock an association, you have to modify [the imported file](#import-api). Before doing any modifications, create a local copy and add it to your source code management system. ```sh cp srv/external/API_BUSINESS_PARTNER.cds srv/external/API_BUSINESS_PARTNER-orig.cds git add srv/external/API_BUSINESS_PARTNER-orig.cds ... ``` Import the CDS file again, just using a different name: ```sh cds import ~/Downloads/API_BUSINESS_PARTNER.edmx --keep-namespace \ --as cds --out srv/external/API_BUSINESS_PARTNER-new.cds ``` Add an `on` condition to express the relation: ::: code-group ```cds [srv/external/API_BUSINESS_PARTNER-new.cds] entity API_BUSINESS_PARTNER.A_BusinessPartner { // ... to_BusinessPartnerAddress : Association to many API_BUSINESS_PARTNER.A_BusinessPartnerAddress on to_BusinessPartnerAddress.BusinessPartner = BusinessPartner; }; ``` ::: Don't add any keys or remove empty keys, which would change it to a managed association. Added fields aren't known in the service and lead to runtime errors. Use a 3-way merge tool to take over your modifications, check it and overwrite the previous unmodified file with the newly imported file: ```sh git merge-file API_BUSINESS_PARTNER.cds \ API_BUSINESS_PARTNER-orig.cds \ API_BUSINESS_PARTNER-new.cds mv API_BUSINESS_PARTNER-new.cds API_BUSINESS_PARTNER-orig.cds ``` To prevent accidental loss of modifications, the `cds import --as cds` command refuses to overwrite modified files based on a "checksum" that is included in the file. ### Mock Remote Service as OData Service (Node.js) {.node} As shown previously you can run one process including a mocked external service. However, this mock doesn't behave like a real external service. The communication happens in-process and doesn't use HTTP or OData. For a more realistic testing, let the mocked service run in a separate process. First install the required packages: ```sh npm add @sap-cloud-sdk/http-client@3.x @sap-cloud-sdk/connectivity@3.x @sap-cloud-sdk/resilience@3.x ``` Then start the CAP application with the mocked remote service only: ```sh cds mock API_BUSINESS_PARTNER ``` If the startup is completed, run `cds watch` in the same project from a **different** terminal: ```sh cds watch ``` CAP tracks locally running services. The mocked service `API_BUSINESS_PARTNER` is registered in file _~/.cds-services.json_. `cds watch` searches for running services in that file and connects to them. Node.js only supports *OData V4* protocol and so does the mocked service. There might still be some differences to the real remote service if it uses a different protocol, but it's much closer to it than using only one instance. In the console output, you can also easily see how the communication between the two processes happens. ### Mock Remote Service as OData Service (Java) {.java} You configure CAP to do OData and HTTP requests for a mocked service instead of doing it in-process. Configure a new Spring Boot profile (for example `mocked`): ::: code-group ```yaml [srv/src/main/resources/application.yaml] spring: config.activate.on-profile: mocked cds: application.services: - name: API_BUSINESS_PARTNER-mocked model: API_BUSINESS_PARTNER serve.path: API_BUSINESS_PARTNER remote.services: API_BUSINESS_PARTNER: destination: name: "s4-business-partner-api-mocked" ``` ::: The profile exposes the mocked service as OData service and defines a destination to access the service. The destination just points to the CAP application itself. You need to implement some Java code for this: ::: code-group ```java [DestinationConfiguration.java] @EventListener void applicationReady(ApplicationReadyEvent ready) { int port = Integer.valueOf(environment.getProperty("local.server.port")); DefaultHttpDestination mockDestination = DefaultHttpDestination .builder("http://localhost:" + port) .name("s4-business-partner-api-mocked").build(); DefaultDestinationLoader loader = new DefaultDestinationLoader(); loader.registerDestination(mockDestination); DestinationAccessor.prependDestinationLoader(loader); } ``` ::: Now, you just need to run the application with the new profile: ```sh mvn spring-boot:run -Dspring-boot.run.profiles=default,mocked ``` When sending a request to your CAP application, for example the `Suppliers` entity, it is transformed to the request for the mocked remote service and requested from itself as a OData request. Therefore, you'll see two HTTP requests in your CAP application's log. For example: [http://localhost:8080/service/risk/Suppliers](http://localhost:8080/service/risk/Suppliers) ```log 2021-09-21 15:18:44.870 DEBUG 34645 — [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : GET "/service/risk/Suppliers", parameters={} ... 2021-09-21 15:18:45.292 DEBUG 34645 — [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet : GET "/API_BUSINESS_PARTNER/A_BusinessPartner?$select=BusinessPartner,BusinessPartnerFullName,BusinessPartnerIsBlocked&$top=1000&$skip=0&$orderby=BusinessPartner%20asc&sap-language=de&sap-valid-at=2021-09-21T13:18:45.211722Z", parameters={masked} ... 2021-09-21 15:18:45.474 DEBUG 34645 — [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet : Completed 200 OK 2021-09-21 15:18:45.519 DEBUG 34645 — [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed 200 OK ``` [Try out the example application.](https://github.com/SAP-samples/cloud-cap-risk-management/tree/ext-service-s4hc-suppliers-ui-java){.learn-more} ## Execute Queries {#execute-queries} You can send requests to remote services using CAP's powerful querying API. ### Execute Queries with Node.js{.node} Connect to the service before sending a request, as usual in CAP: ```js const bupa = await cds.connect.to('API_BUSINESS_PARTNER'); ``` Then execute your queries using the [Querying API](../node.js/core-services#srv-run-query): ```js const { A_BusinessPartner } = bupa.entities; const result = await bupa.run(SELECT(A_BusinessPartner).limit(100)); ``` We recommend limiting the result set and avoid the download of large data sets in a single request. You can `limit` the result as in the example: `.limit(100)`. Many features of the querying API are supported for OData services. For example, you can resolve associations like this: ```js const { A_BusinessPartner } = bupa.entities; const result = await bupa.run(SELECT.from(A_BusinessPartner, bp => { bp('BusinessPartner'), bp.to_BusinessPartnerAddress(addresses => { addresses('*') }) }).limit(100)); ``` [Learn more about querying API examples.](https://github.com/SAP-samples/cloud-cap-risk-management/blob/ext-service-s4hc-suppliers-ui/test/odata-examples.js){.learn-more} [Learn more about supported querying API features.](#querying-api-features){.learn-more} ### Execute Queries with Java {.java} You can use dependency injection to get access to the remote service: ```java @Autowired @Qualifier(ApiBusinessPartner_.CDS_NAME) CqnService bupa; ``` Then execute your queries using the [Querying API](../java/working-with-cql/query-execution): ```java CqnSelect select = Select.from(ABusinessPartner_.class).limit(100); List businessPartner = bupa.run(select).listOf(ABusinessPartner.class); ``` [Learn more about querying API examples.](https://github.com/SAP-samples/cloud-cap-risk-management/blob/ext-service-s4hc-suppliers-ui/test/odata-examples.js){.learn-more} [Learn more about supported querying API features.](#querying-api-features){.learn-more} ### Model Projections External service definitions, like [generated CDS or CSN files during import](#import-api), can be used as any other CDS definition, but they **don't** generate database tables and views unless they are mocked. It's best practice to use your own "interface" to the external service and define the relevant fields in a projection in your namespace. Your implementation is then independent of the remote service implementation and you request only the information that you require. ```cds using { API_BUSINESS_PARTNER as bupa } from '../srv/external/API_BUSINESS_PARTNER'; entity Suppliers as projection on bupa.A_BusinessPartner { key BusinessPartner as ID, BusinessPartnerFullName as fullName, BusinessPartnerIsBlocked as isBlocked, } ``` As the example shows, you can use field aliases as well. [Learn more about supported features for projections.](#supported-projection-features){.learn-more} ### Execute Queries on Projections to a Remote Service{.node} Connect to the service before sending a request, as usual in CAP: ```js const bupa = await cds.connect.to('API_BUSINESS_PARTNER'); ``` Then execute your queries: ```js const suppliers = await bupa.run(SELECT(Suppliers).where({ID})); ``` CAP resolves projections and does the required mapping, similar to databases. A brief explanation, based on the previous query, what CAP does: + Resolves the `Suppliers` projection to the external service interface `API_BUSINESS_PARTNER.A_Business_Partner`. + The **where** condition for field `ID` will be mapped to the `BusinessPartner` field of `A_BusinessPartner`. + The result is mapped back to the `Suppliers` projection, so that values for the `BusinessPartner` field are mapped back to `ID`. This makes it convenient to work with external services. ### Building Custom Requests with Node.js{.node} If you can't use the querying API, you can craft your own HTTP requests using `send`: ```js bupa.send({ method: 'PATCH', path: A_BusinessPartner, data: { BusinessPartner: 1004155, BusinessPartnerIsBlocked: true } }) ``` [Learn more about the `send` API.](../node.js/core-services#srv-send-request){.learn-more} ### Building Custom Requests with Java {.java} For Java, you can use the `HttpClient` API to implement your custom requests. The API is enhanced by the SAP Cloud SDK to support destinations. [Learn more about using the HttpClient Accessor.](https://sap.github.io/cloud-sdk/docs/java/features/connectivity/sdk-connectivity-http-client){.learn-more} [Learn more about using destinations.](#use-destinations-with-java){.learn-more} ## Integrate and Extend {#integrate-and-extend} By creating projections on remote service entities and using associations, you can create services that combine data from your local service and remote services. What you need to do depends on [the scenarios](#sample-scenario-from-end-to-end-tutorial) and how your remote services should be integrated into, as well as extended by your local services. ### Expose Remote Services To expose a remote service entity, you add a projection on it to your CAP service: ```cds using { API_BUSINESS_PARTNER as bupa } from '../srv/external/API_BUSINESS_PARTNER'; extend service RiskService with { entity BusinessPartners as projection on bupa.A_BusinessPartner; } ``` CAP automatically tries to delegate queries to database entities, which don't exist as you're pointing to an external service. That behavior would produce an error like this: ```xml 500 SQLITE_ERROR: no such table: RiskService_BusinessPartners in: SELECT BusinessPartner, Customer, Supplier, AcademicTitle, AuthorizationGroup, BusinessPartnerCategory, BusinessPartnerFullName, BusinessPartnerGrouping, BusinessPartnerName, BusinessPartnerUUID, CorrespondenceLanguage, CreatedByUser, CreationDate, (...) FROM RiskService_BusinessPartner ALIAS_1 ORDER BY BusinessPartner COLLATE NOCASE ASC LIMIT 11 ``` To avoid this error, you need to handle projections. Write a handler function to delegate a query to the remote service and run the incoming query on the external service. ::: code-group ```js [Node.js] module.exports = cds.service.impl(async function() { const bupa = await cds.connect.to('API_BUSINESS_PARTNER'); this.on('READ', 'BusinessPartners', req => { return bupa.run(req.query); }); }); ``` ```java [Java] @Component @ServiceName(RiskService_.CDS_NAME) public class RiskServiceHandler implements EventHandler { @Autowired @Qualifier(ApiBusinessPartner_.CDS_NAME) CqnService bupa; @On(entity = BusinessPartners.CDS_NAME) Result readSuppliers(CdsReadEventContext context) { return bupa.run(context.getCqn()); } } ``` ::: [For Node.js, get more details in the end-to-end tutorial.](https://developers.sap.com/tutorials/btp-app-ext-service-add-consumption.html#0a5ed8cc-d0fa-4a52-bb56-9c864cd66e71){.learn-more} ::: warning If you receive `404` errors, check if the request contains fields that don't exist in the service and start with the name of an association. `cds import` adds an empty keys declaration (`{ }`) to each association. Without this declaration, foreign keys for associations are generated in the runtime model, that don't exist in the real service. To solve this problem, you need to reimport the external service definition using `cds import`. ::: This works when accessing the entity directly. Additional work is required to support [navigation](#handle-navigations-across-local-and-remote-entities) and [expands](#handle-expands-across-local-and-remote-entities) from or to a remote entity. Instead of exposing the remote service's entity unchanged, you can [model your own projection](#model-projections). For example, you can define a subset of fields and change their names. ::: tip CAP does the magic that maps the incoming query, according to your projections, to the remote service and maps back the result. ::: ```cds using { API_BUSINESS_PARTNER as bupa } from '../srv/external/API_BUSINESS_PARTNER'; extend service RiskService with { entity Suppliers as projection on bupa.A_BusinessPartner { key BusinessPartner as ID, BusinessPartnerFullName as fullName, BusinessPartnerIsBlocked as isBlocked } } ``` ```js module.exports = cds.service.impl(async function() { const bupa = await cds.connect.to('API_BUSINESS_PARTNER'); this.on('READ', 'Suppliers', req => { return bupa.run(req.query); }); }); ``` [Learn more about queries on projections to remote services.](#execute-queries-on-projections-to-a-remote-service){.learn-more} ### Expose Remote Services with Associations It's possible to expose associations of a remote service entity. You can adjust the [projection for the association target](#model-projections) and change the name of the association: ```cds using { API_BUSINESS_PARTNER as bupa } from '../srv/external/API_BUSINESS_PARTNER'; extend service RiskService with { entity Suppliers as projection on bupa.A_BusinessPartner { key BusinessPartner as ID, BusinessPartnerFullName as fullName, BusinessPartnerIsBlocked as isBlocked, to_BusinessPartnerAddress as addresses: redirected to SupplierAddresses } entity SupplierAddresses as projection on bupa.A_BusinessPartnerAddress { BusinessPartner as bupaID, AddressID as ID, CityName as city, StreetName as street, County as county } } ``` As long as the association is only resolved using expands (for example `.../risk/Suppliers?$expand=addresses`), a handler for the __source entity__ is sufficient: ```js this.on('READ', 'Suppliers', req => { return bupa.run(req.query); }); ``` If you need to resolve the association using navigation or request it independently from the source entity, add a handler for the __target entity__ as well: ```js this.on('READ', 'SupplierAddresses', req => { return bupa.run(req.query); }); ``` As usual, you can put two handlers into one handler matching both entities: ```js this.on('READ', ['Suppliers', 'SupplierAddresses'], req => { return bupa.run(req.query); }); ``` ### Mashing up with Remote Services You can combine local and remote services using associations. These associations need manual handling, because of their different data sources. #### Integrate Remote into Local Services Use managed associations from local entities to remote entities: ```cds @path: 'service/risk' service RiskService { entity Risks : managed { key ID : UUID @(Core.Computed : true); title : String(100); prio : String(5); supplier : Association to Suppliers; } entity Suppliers as projection on BusinessPartner.A_BusinessPartner { key BusinessPartner as ID, BusinessPartnerFullName as fullName, BusinessPartnerIsBlocked as isBlocked, }; } ``` #### Extend a Remote by a Local Service { #extend-a-remote-by-a-local-service} You can augment a projection with a new association, if the required fields for the on condition are present in the remote service. The use of managed associations isn't possible, because this requires to create new fields in the remote service. ```cds entity Suppliers as projection on bupa.A_BusinessPartner { key BusinessPartner as ID, BusinessPartnerFullName as fullName, BusinessPartnerIsBlocked as isBlocked, risks : Association to many Risks on risks.supplier.ID = ID, }; ``` ### Handle Mashups with Remote Services { #building-mashups} Depending on how the service is accessed, you need to support direct requests, navigation, or expands. CAP resolves those three request types only for service entities that are served from the database. When crossing the boundary between database and remote sourced entities, you need to take care of those requests. The list of [required implementations for mashups](#required-implementations-for-mashups) explains the different combinations. #### Handle Expands Across Local and Remote Entities Expands add data from associated entities to the response. For example, for a risk, you want to display the suppliers name instead of just the technical ID. But this property is part of the (remote) supplier and not part of the (local) risk. To handle expands, you need to add a handler for the main entity: 1. Check if a relevant `$expand` column is present. 2. Remove the `$expand` column from the request. 3. Get the data for the request. 4. Execute a new request for the expand. 5. Add the expand data to the returned data from the request. Example of a CQN request with an expand: ```json { "from": { "ref": [ "RiskService.Suppliers" ] }, "columns": [ { "ref": [ "ID" ] }, { "ref": [ "fullName" ] }, { "ref": [ "isBlocked" ] }, { "ref": [ "risks" ] }, { "expand": [ { "ref": [ "ID" ] }, { "ref": [ "title" ] }, { "ref": [ "descr" ] }, { "ref": [ "supplier_ID" ] } ] } ] } ``` [See an example how to handle expands in Node.js.](https://github.com/SAP-samples/cloud-cap-risk-management/blob/ext-service-s4hc-suppliers-ui/srv/risk-service.js){.node .learn-more} [See an example how to handle expands in Java.](https://github.com/SAP-samples/cloud-cap-risk-management/blob/ext-service-s4hc-suppliers-ui-java/srv/src/main/java/com/sap/cap/riskmanagement/handler/RiskServiceHandler.java){.java .learn-more} Expands across local and remote can cause stability and performance issues. For a list of items, you need to collect all IDs and send it to the database or the remote system. This can become long and may exceed the limits of a URL string in case of OData. Do you really need expands for a list of items? ```http GET /service/risk/Risks?$expand=supplier ``` Or is it sufficient for single items? ```http GET /service/risk/Risks(545A3CF9-84CF-46C8-93DC-E29F0F2BC6BE)/?$expand=supplier ``` ::: warning Keep performance in mind Consider to reject expands if it's requested on a list of items. ::: #### Handle Navigations Across Local and Remote Entities Navigations allow to address items via an association from a different entity: ```http GET /service/risks/Risks(20466922-7d57-4e76-b14c-e53fd97dcb11)/supplier ``` The CQN consists of a `from` condition with 2 values for `ref`. The first `ref` selects the record of the source entity of the navigation. The second `ref` selects the name of the association, to navigate to the target entity. ```json { "from": { "ref": [ { "id": "RiskService.Risks", "where": [ { "ref": [ "ID" ] }, "=", { "val": "20466922-7d57-4e76-b14c-e53fd97dcb11" } ]}, "supplier" ] }, "columns": [ { "ref": [ "ID" ] }, { "ref": [ "fullName" ] }, { "ref": [ "isBlocked" ] } ], "one": true } ``` To handle navigations, you need to check in your code if the `from.ref` object contains 2 elements. Be aware, that for navigations the handler of the **target** entity is called. If the association's on condition equals the key of the source entity, you can directly select the target entity using the key's value. You find the value in the `where` block of the first `from.ref` entry. Otherwise, you need to select the source item using that `where` block and take the required fields for the associations on condition from that result. [See an example how to handle navigations in Node.js.](https://github.com/SAP-samples/cloud-cap-risk-management/blob/ext-service-s4hc-suppliers-ui/srv/risk-service.js){.learn-more .node} [See an example how to handle navigations in Java.](https://github.com/SAP-samples/cloud-cap-risk-management/blob/ext-service-s4hc-suppliers-ui-java/srv/src/main/java/com/sap/cap/riskmanagement/handler/RiskServiceHandler.java){.learn-more .java} ### Limitations and Feature Matrix #### Required Implementations for Mashups { #required-implementations-for-mashups} You need additional logic, if remote entities are in the game. The following table shows what is required. "Local" is a database entity or a projection on a database entity. | **Request** | **Example** | **Implementation** | | --------------------------------------------------------------------- | ---------------------------------------- | ----------------------------------------------------------------- | | Local (including navigations and expands) | `/service/risks/Risks` | Handled by CAP | | Local: Expand remote | `/service/risks/Risks?$expand=supplier` | Delegate query w/o expand to local service and implement expand. | | Local: Navigate to remote | `/service/risks(...)/supplier` | Implement navigation and delegate query target to remote service. | | Remote (including navigations and expands to the same remote service) | `/service/risks/Suppliers` | Delegate query to remote service | | Remote: Expand local | `/service/risks/Suppliers?$expand=risks` | Delegate query w/o expand to remote service and implement expand. | | Remote: Navigate to local | `/service/Suppliers(...)/risks` | Implement navigation, delegate query for target to local service | #### Transient Access vs. Replication > This chapter shows only techniques for transient access. The following matrix can help you to find the best approach for your scenario: | **Feature** | **Transient Access** | **Replication** | |-------------------------------------------------------|-----------------------|-----------------------------------| | Filtering on local **or** remote fields ¹ | Possible | Possible | | Filtering on local **and** remote fields ² | Not possible | Possible | | Relationship: Uni-/Bidirectional associations | Possible | Possible | | Relationship: Flatten | Not possible | Possible | | Evaluate user permissions in remote system | Possible | Requires workarounds ³ | | Data freshness | Live data | Outdated until replicated | | Performance | Degraded ⁴ | Best |
> ¹ It's **not required** to filter both, on local and remote fields, in the same request.
> ² It's **required** to filter both, on local and remote fields, in the same request.
> ³ Because replicated data is accessed, the user permission checks of the remote system aren't evaluated.
> ⁴ Depends on the connectivity and performance of the remote system.
## Connect and Deploy {#connect-and-deploy} ### Using Destinations { #using-destinations} Destinations contain the necessary information to connect to a remote system. They're basically an advanced URL, that can carry additional metadata like, for example, the authentication information. You can choose to use [SAP BTP destinations](#btp-destinations) or [application defined destinations](#app-defined-destinations). #### Use SAP BTP Destinations { #btp-destinations} CAP leverages the destination capabilities of the SAP Cloud SDK. ##### Create Destinations on SAP BTP Create a destination using one or more of the following options. - **Register a system in your global account:** You can check here how to [Register an SAP System](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/2ffdaff0f1454acdb046876045321c91.html) in your SAP BTP global account and which systems are supported for registration. Once the system is registered and assigned to your subaccount, you can create a service instance. A destination is automatically created along with the service instance. - **Connect to an on-premise system:** With SAP BTP [Cloud Connector](https://help.sap.com/docs/CP_CONNECTIVITY/cca91383641e40ffbe03bdc78f00f681/e6c7616abb5710148cfcf3e75d96d596.html), you can create a connection from your cloud application to an on-premise system. - **Manually create destinations:** You can create destinations manually in your SAP BTP subaccount. See section [destinations](https://help.sap.com/docs/CP_CONNECTIVITY/cca91383641e40ffbe03bdc78f00f681/5eba6234a0e143fdacd8535f44c315c5.html) in the SAP BTP documentation. - **Create a destination to your application:** If you need a destination to your application, for example, to call it from a different application, then you can automatically create it in the MTA deployment. ##### Use Destinations with Node.js {.node} In your _package.json_, a configuration for the `API_BUSINESS_PARTNER` looks like this: ```json "cds": { "requires": { "API_BUSINESS_PARTNER": { "kind": "odata", "model": "srv/external/API_BUSINESS_PARTNER" } } } ``` If you've imported the external service definition using `cds import`, an entry for the service in the _package.json_ has been created already. Here you specify the name of the destination in the `credentials` block. In many cases, you also need to specify the `path` prefix to the service, which is added to the destination's URL. For services listed on the SAP Business Accelerator Hub, you can find the path in the linked service documentation. Since you don't want to use the destination for local testing, but only for production, you can profile it by wrapping it into a `[production]` block: ```json "cds": { "requires": { "API_BUSINESS_PARTNER": { "kind": "odata", "model": "srv/external/API_BUSINESS_PARTNER", "[production]": { "credentials": { "destination": "S4HANA", "path": "/sap/opu/odata/sap/API_BUSINESS_PARTNER" } } } } } ``` Additionally, you can provide [destination options](https://sap.github.io/cloud-sdk/api/v3/types/sap_cloud_sdk_connectivity.DestinationOptions.html) inside a `destinationOptions` object: ```jsonc "cds": { "requires": { "API_BUSINESS_PARTNER": { /* ... */ "[production]": { "credentials": { /* ... */ }, "destinationOptions": { "selectionStrategy": "alwaysSubscriber", "useCache": true } } } } } ``` The `selectionStrategy` property controls how a [destination is resolved](#destination-resolution). The `useCache` option controls whether the SAP Cloud SDK caches the destination. It's enabled by default but can be disabled by explicitly setting it to `false`. Read [Destination Cache](https://sap.github.io/cloud-sdk/docs/js/features/connectivity/destination-cache#destination-cache) to learn more about how the cache works. If you want to configure additional headers for the HTTP request to the system behind the destination, for example an Application Interface Register (AIR) header, you can specify such headers in the destination definition itself using the property [_URL.headers.\_](https://help.sap.com/docs/CP_CONNECTIVITY/cca91383641e40ffbe03bdc78f00f681/4e1d742a3d45472d83b411e141729795.html?q=URL.headers). ##### Use Destinations with Java {.java} Destinations are configured in Spring Boot's _application.yaml_ file: ::: code-group ```yaml [srv/src/main/resources/application.yaml] cds: remote.services: API_BUSINESS_PARTNER: type: "odata-v2" destination: name: "cpapp-bupa" http: suffix: "/sap/opu/odata/sap" ``` ::: [Learn more about configuring destinations for Java.](../java/cqn-services/remote-services#destination-based-scenarios){.learn-more} #### Use Application Defined Destinations { #app-defined-destinations} If you don't want to use SAP BTP destinations, you can also define destinations, which means the URL, authentication type, and additional configuration properties, in your application configuration or code. Application defined destinations support a subset of [properties](#destination-properties) and [authentication types](#authentication-types) of the SAP BTP destination service. ##### Configure Application Defined Destinations in Node.js {.node} You specify the destination properties in `credentials` instead of putting the name of a destination there. This is an example of a destination using basic authentication: ```jsonc "cds": { "requires": { "REVIEWS": { "kind": "odata", "model": "srv/external/REVIEWS", "[production]": { "credentials": { "url": "https://reviews.ondemand.com/reviews", "authentication": "BasicAuthentication", "username": "", "password": "", "headers": { "my-header": "header value" }, "queries": { "my-url-param": "url param value" } } } } } } ``` [Supported destination properties.](#destination-properties){.learn-more} ::: warning You shouldn't put any sensitive information here. ::: Instead, set the properties in the bootstrap code of your CAP application: ```js const cds = require("@sap/cds"); if (cds.env.requires?.credentials?.authentication === "BasicAuthentication") { const credentials = /* read your credentials */ cds.env.requires.credentials.username = credentials.username; cds.env.requires.credentials.password = credentials.password; } ``` You might also want to set some values in the application deployment. This can be done using env variables. For this example, the env variable for the URL would be `cds_requires_REVIEWS_credentials_destination_url`. This variable can be parameterized in the _manifest.yml_ for a `cf push` based deployment: ::: code-group ```yaml [manifest.yml] applications: - name: reviews ... env: cds_requires_REVIEWS_credentials_url: ((reviews_url)) ``` ::: ```sh cf push --var reviews_url=https://reviews.ondemand.com/reviews ``` The same can be done using _mtaext_ file for MTA deployment. If the URL of the target service is also part of the MTA deployment, you can automatically receive it as shown in this example: ::: code-group ```yaml [mta.yaml] - name: reviews provides: - name: reviews-api properties: reviews-url: ${default-url} - name: bookshop requires: ... - name: reviews-api properties: cds_requires_REVIEWS_credentials_url: ~{reviews-api/reviews-url} ``` ::: ::: code-group ```properties [.env] cds_requires_REVIEWS_credentials_url=http://localhost:4008/reviews ``` ::: ::: warning For the _configuration path_, you **must** use the underscore ("`_`") character as delimiter. CAP supports dot ("`.`") as well, but Cloud Foundry won't recognize variables using dots. Your _service name_ **mustn't** contain underscores. ::: ##### Implement Application Defined Destinations in Node.js {.node} There is no API to create a destination in Node.js programmatically. However, you can change the properties of a remote service before connecting to it, as shown in the previous example. ##### Configure Application Defined Destinations in Java {.java} Destinations are configured in Spring Boot's _application.yaml_ file. ::: code-group ```yaml [srv/src/main/resources/application.yaml] cds: remote.services: REVIEWS: type: "odata-v4" destination: properties: url: https://reviews.ondemand.com/reviews authentication: TokenForwarding http: headers: my-header: "header value" queries: my-url-param: "url param value" ``` ::: [Learn more about supported destination properties.](#destination-properties){.learn-more} ##### Implement Application Defined Destinations in Java {.java} You can use the APIs offered by SAP Cloud SDK to create destinations programmatically. The destination can be used by its name the same way as destinations on the SAP BTP destination service. ::: code-group ```yaml [srv/src/main/resources/application.yaml] cds: remote.services: REVIEWS: type: "odata-v2" destination: name: "reviews-destination" ``` ::: [Learn more about programmatic destination registration.](../java/cqn-services/remote-services#programmatic-destination-registration){.learn-more} [See examples for different authentication types.](../java/cqn-services/remote-services#programmatic-destinations){.learn-more} ### Connect to Remote Services Locally If you use SAP BTP destinations, you can access them locally using [CAP's hybrid testing capabilities](../advanced/hybrid-testing) with the following procedure: #### Bind to Remote Destinations Your local application needs access to an XSUAA and Destination service instance in the same subaccount where the destination is: 1. Login to your Cloud Foundry org and space 2. Create an XSUAA service instance and service key: ```sh cf create-service xsuaa application cpapp-xsuaa cf create-service-key cpapp-xsuaa cpapp-xsuaa-key ``` 3. Create a Destination service instance and service key: ```sh cf create-service destination lite cpapp-destination cf create-service-key cpapp-destination cpapp-destination-key ``` 4. Bind to XSUAA and Destination service: ```sh cds bind -2 cpapp-xsuaa,cpapp-destination ``` [Learn more about `cds bind`.](../advanced/hybrid-testing#services-on-cloud-foundry){.learn-more} #### Run a Node.js Application with a Destination {.node} Add the destination for the remote service to the `hybrid` profile in the _.cdsrc-private.json_ file: ```jsonc { "requires": { "[hybrid]": { "auth": { /* ... */ }, "destinations": { /* ... */ }, "API_BUSINESS_PARTNER": { "credentials": { "path": "/sap/opu/odata/sap/API_BUSINESS_PARTNER", "destination": "cpapp-bupa" } } } } } ``` Run your application with the Destination service: ```sh cds watch --profile hybrid ``` ::: tip If you are developing in the Business Application Studio and want to connect to an on-premise system, you will need to do so via Business Application Studio's built-in proxy, for which you need to add configuration in an `.env` file. See [Connecting to External Systems From the Business Application Studio](https://sap.github.io/cloud-sdk/docs/js/guides/bas) for more details. ::: #### Run a Java Application with a Destination {.java} Add a new profile `hybrid` to your _application.yaml_ file that configures the destination for the remote service. ::: code-group ```yaml [srv/src/main/resources/application.yaml] spring: config.activate.on-profile: hybrid sql.init.schema-locations: - "classpath:schema-nomocks.sql" cds: remote.services: - name: API_BUSINESS_PARTNER type: "odata-v2" destination: name: "cpapp-bupa" http: suffix: "/sap/opu/odata/sap" ``` ::: Run your application with the Destination service: ```sh cds bind --exec -- mvn spring-boot:run \ -Dspring-boot.run.profiles=default,hybrid ``` [Learn more about `cds bind --exec`.](../advanced/hybrid-testing#run-arbitrary-commands-with-service-bindings){.learn-more} ::: tip If you are developing in the Business Application Studio and want to connect to an on-premise system, you will need to do so via Business Application Studio's built-in proxy, for which you need to add configuration to your destination environment variable. See [Reach On-Premise Service from the SAP Business Application Studio](https://sap.github.io/cloud-sdk/docs/java/features/connectivity/destination-service#reach-on-premise-service-from-the-sap-business-application-studio) for more details. ::: ### Connect to an Application Using the Same XSUAA (Forward Authorization Token) {#forward-auth-token} If your application consists of microservices and you use one (or more) as a remote service as described in this guide, you can leverage the same XSUAA instance. In that case, you don't need an SAP BTP destination at all. Assuming that your microservices use the same XSUAA instance, you can just forward the authorization token. The URL of the remote service can be injected into the application in the [MTA or Cloud Foundry deployment](#deployment) using [application defined destinations](#app-defined-destinations). #### Forward Authorization Token with Node.js{.node} To enable the token forwarding, set the `forwardAuthToken` option to `true` in your application defined destination: ```json { "requires": { "kind": "odata", "model": "./srv/external/OrdersService", "credentials": { "url": "", "forwardAuthToken": true } } } ``` #### Forward Authorization Token with Java{.java} For Java, you set the authentication type to `TOKEN_FORWARDING` for the destination. You can implement it in your code: ```java urlFromConfig = ...; // read from config DefaultHttpDestination mockDestination = DefaultHttpDestination .builder(urlFromConfig) .name("order-service") .authenticationType(AuthenticationType.TOKEN_FORWARDING) .build(); ``` Or declare the destination in your _application.yaml_ file: ::: code-group ```yaml [srv/src/main/resources/application.yaml] cds: remote.services: order-service: type: "odata-v4" destination: properties: url: "" authentication: TokenForwarding ``` ::: Alternatively to setting the authentication type, you can set the property `forwardAuthToken` to `true`. ### Connect to an Application in Your Kyma Cluster The [Istio](https://istio.io) service mesh provides secure communication between the services in your service mesh. You can access a service in your applications' namespace by just reaching out to `http://` or using the full hostname `http://..svc.cluster.local`. Istio sends the requests through an mTLS tunnel. With Istio, you can further secure the communication [by configuring authentication and authorization for your services](https://istio.io/latest/docs/concepts/security) ### Deployment Your micro service needs bindings to the **XSUAA** and **Destination** service to access destinations on SAP BTP. If you want to access an on-premise service using **Cloud Connector**, then you need a binding to the **Connectivity** service as well. [Learn more about deploying CAP applications.](deployment/){.learn-more} [Learn more about deploying an application using the end-to-end tutorial.](https://developers.sap.com/group.btp-app-cap-deploy.html){.learn-more} #### Add Required Services to MTA Deployments The MTA-based deployment is described in [the deployment guide](deployment/). You can follow this guide and make some additional adjustments to the [generated _mta.yml_](deployment/to-cf#add-mta-yaml) file. ```sh cds add xsuaa,destination,connectivity --for production ``` ::: details Learn what this does in the background... 1. Adds **XSUAA**, **Destination**, and **Connectivity** services to your _mta.yaml_: ::: code-group ```yaml [mta.yml] - name: cpapp-uaa type: org.cloudfoundry.managed-service parameters: service: xsuaa service-plan: application path: ./xs-security.json - name: cpapp-destination type: org.cloudfoundry.managed-service parameters: service: destination service-plan: lite # Required for on-premise connectivity only - name: cpapp-connectivity type: org.cloudfoundry.managed-service parameters: service: connectivity service-plan: lite ``` ::: 2. Requires the services for your server in the _mta.yaml_: ::: code-group ```yaml [mta.yaml] - name: cpapp-srv ... requires: ... - name: cpapp-uaa - name: cpapp-destination - name: cpapp-connectivity # Required for on-premise connectivity only ``` ::: ::: Build your application: ```sh mbt build -t gen --mtar mta.tar ``` Now you can deploy it to Cloud Foundry: ```sh cf deploy gen/mta.tar ``` #### Connectivity Service Credentials on Kyma The secret of the connectivity service on Kyma needs to be modified for the Cloud SDK to connect to on-premise destinations. [Support for Connectivity Service Secret in Java](https://github.com/SAP/cloud-sdk/issues/657){.java .learn-more} [Support for Connectivity Service Secret in Node.js](https://github.com/SAP/cloud-sdk-js/issues/2024){.node .learn-more} ### Destinations and Multitenancy With the destination service, you can access destinations in your provider account, the account your application is running in, and destinations in the subscriber accounts of your multitenant-aware application. #### Use Destinations from Subscriber Account Customers want to see business partners from, for example, their SAP S/4 HANA system. As provider, you need to define a name for a destination, which enables access to systems of the subscriber of your application. In addition, your multitenant application or service needs to have a dependency to the destination service. For destinations in an on-premise system, the connectivity service must be bound. The subscriber needs to create a destination with that name in their subscriber account, for example, pointing to their SAP S/4HANA system. #### Destination Resolution The destination is read from the tenant of the request's JWT (authorization) token. If no JWT token is present, the destination is read from the tenant of the application's XSUAA binding.{.java} The destination is read from the tenant of the request's JWT (authorization) token. If no JWT token is present *or the destination isn't found*, the destination is read from the tenant of the application's XSUAA binding.{.node} ::: warning JWT token vs. XSUAA binding Using the tenant of the request's JWT token means reading from the **subscriber subaccount** for a multitenant application. The tenant of the application's XSUAA binding points to the destination of the **provider subaccount**, the account where the application is deployed to. :::

You can change the destination lookup behavior as follows: ```jsonc "cds": { "requires": { "SERVICE_FOR_PROVIDER": { /* ... */ "credentials": { /* ... */ }, "destinationOptions": { "selectionStrategy": "alwaysProvider", "jwt": null } } } } ``` Setting the [`selectionStrategy`](https://sap.github.io/cloud-sdk/docs/js/features/connectivity/destination#multi-tenancy) property for the [destination options](#use-destinations-with-node-js) to `alwaysProvider`, you can ensure that the destination is always read from your provider subaccount. With that you ensure that a subscriber cannot overwrite your destination. Set the destination option `jwt` to `null`, if you don't want to pass the request's JWT to SAP Cloud SDK. Passing the request's JWT to SAP Cloud SDK has implications on, amongst others, the effective defaults for selection strategy and isolation level. In rare cases, these defaults are not suitable, for example when the request to the upstream server does not depend on the current user. Please see [Authentication and JSON Web Token (JWT) Retrieval](https://sap.github.io/cloud-sdk/docs/js/features/connectivity/destinations#authentication-and-json-web-token-jwt-retrieval) for more details.

For Java use the property `retrievalStrategy` in the destination configuration, to ensure that the destination is always read from your provider subaccount: ```yaml cds: remote.services: service-for-provider: type: "odata-v4" destination: retrievalStrategy: "AlwaysProvider" ``` Read more in the full reference of all [supported retrieval strategy values](https://sap.github.io/cloud-sdk/docs/java/features/connectivity/sdk-connectivity-destination-service#retrieval-strategy-options). Please note that the value must be provided in pascal case, for example: `AlwaysProvider`.

## Add Qualities

### Resilience There are two ways to make your outbound communications resilient: 1. Run your application in a service mesh (for example, Istio, Linkerd, etc.). For example, [Kyma is provided as service mesh](#resilience-in-kyma). 2. Implement resilience in your application. Refer to the documentation for the service mesh of your choice for instructions. No code changes should be required. To build resilience into your application, there are libraries to help you implement functions, like doing retries, circuit breakers or implementing fallbacks.

You can use the [resilience features](https://sap.github.io/cloud-sdk/docs/java/features/resilience) provided by the SAP Cloud SDK with CAP Java. You need to wrap your remote calls with a call of `ResilienceDecorator.executeSupplier` and a resilience configuration (`ResilienceConfiguration`). Additionally, you can provide a fallback function. ```java ResilienceConfiguration config = ResilienceConfiguration.of(AdminServiceAddressHandler.class) .timeLimiterConfiguration(TimeLimiterConfiguration.of(Duration.ofSeconds(10))); context.setResult(ResilienceDecorator.executeSupplier(() -> { // ..to access the S/4 system in a resilient way.. logger.info("Delegating GET Addresses to S/4 service"); return bupa.run(select); }, config, (t) -> { // ..falling back to the already replicated addresses in our own database logger.warn("Falling back to already replicated Addresses"); return db.run(select); })); ``` [See the full example](https://github.com/SAP-samples/cloud-cap-samples-java/blob/main/srv/src/main/java/my/bookshop/handlers/AdminServiceAddressHandler.java){.learn-more}

There's no resilience library provided out of the box for CAP Node.js. However, you can use packages provided by the Node.js community. Usually, they provide a function to wrap your code that adds the resilience logic.

#### Resilience in Kyma Kyma clusters run an [Istio](https://istio.io/) service mesh. Istio allows to [configure resilience](https://istio.io/latest/docs/concepts/traffic-management/#network-resilience-and-testing) for the network destinations of your service mesh. ### Tracing CAP adds headers for request correlation to its outbound requests that allows logging and tracing across micro services. [Learn more about request correlation in Node.js.](../node.js/cds-log#node-observability-correlation){.learn-more .node} [Learn more about request correlation in Java.](../java/operating-applications/observability#correlation-ids){.learn-more .java}

## Feature Details ### Legend | Tag | Explanation | |:----:|---------------| | | supported | | | not supported | ### Supported Protocols | Protocol | Java | Node.js | |----------|:----:|:-------:| | odata-v2 | | | | odata-v4 | | | | rest | | | ::: tip The Node.js runtime supports `odata` as an alias for `odata-v4` as well. ::: ### Querying API Features | Feature | Java | Node.js | |------------------------------------|:----:|:-------:| | READ | | | | INSERT/UPDATE/DELETE | | | | Actions | | | | `columns` | | | | `where` | | | | `orderby` | | | | `limit` (top & skip) | | | | `$apply` (aggregate, groupby, ...) | | | | `$search` (OData v4) | | | | `search` (SAP OData v2 extension) | | | ### Supported Projection Features | Feature | Java | Node.js | |-----------------------------------------------------------|:----:|:-------:| | Resolve projections to remote services | | | | Resolve multiple levels of projections to remote services | | | | Aliases for fields | | | | `excluding` | | | | Resolve associations (within the same remote service) | | | | Redirected associations | | | | Flatten associations | | | | `where` conditions | | | | `order by` | | | | Infix filter for associations | | | | Model Associations with mixins | | | ### Supported Features for Application Defined Destinations The following properties and authentication types are supported for *[application defined destinations](#app-defined-destinations)*: #### Properties { #destination-properties} These destination properties are fully supported by both, the Java and the Node.js runtime. ::: tip This list specifies the properties for application defined destinations. ::: | Properties | Description | | -------------------------- | ----------------------------------------- | | `url` | | | `authentication` | Authentication type | | `username` | User name for BasicAuthentication | | `password` | Password for BasicAuthentication | | `headers` | Map of HTTP headers | | `queries` | Map of URL parameters | | `forwardAuthToken` | [Forward auth token](#forward-auth-token) | [Destination Type in SAP Cloud SDK for JavaScript](https://sap.github.io/cloud-sdk/api/v3/interfaces/sap_cloud_sdk_connectivity.Destination.html){.learn-more .node} [HttpDestination Type in SAP Cloud SDK for Java](https://help.sap.com/doc/82a32040212742019ce79dda40f789b9/1.0/en-US/index.html){.learn-more .java} #### Authentication Types | Authentication Types | Java | Node.js | |-------------------------|:-----------------------------------------------------------------------:|:------------------------------:| | NoAuthentication | | | | BasicAuthentication | | | | TokenForwarding | |
Use `forwardAuthToken` | | OAuth2ClientCredentials | [code only](../java/cqn-services/remote-services#programmatic-destinations) | | | UserTokenAuthentication | [code only](../java/cqn-services/remote-services#programmatic-destinations) | | # Events and Messaging CAP provides intrinsic support for emitting and receiving events. This is complemented by Messaging Services connecting to message brokers to exchange event messages across remote services. ## Ubiquitous Events in CAP {#intro} We're starting with an introduction to the core concepts in CAP. If you want to skip the introduction, you can fast-forward to the samples part starting at [Books Reviews Sample](#books-reviews-sample). ### Intrinsic Eventing in CAP Core As introduced in [About CAP](../../about/best-practices#events), everything happening at runtime is in response to events, and all service implementations take place in [event handlers](../providing-services#event-handlers). All CAP services intrinsically support emitting and reacting to events, as shown in this simple code snippet (you can copy & run it in `cds repl`): ```js let srv = new cds.Service // Receiving Events srv.on ('some event', msg => console.log('1st listener received:', msg)) srv.on ('some event', msg => console.log('2nd listener received:', msg)) // Emitting Events await srv.emit ('some event', { foo:11, bar:'12' }) ``` ::: tip Intrinsic support for events The core of CAP's processing model: all services are event emitters. Events can be sent to them, emitted by them, and event handlers register with them to react to such events. ::: ### Typical Emitter and Receiver Roles In contrast to the previous code sample, emitters and receivers of events are decoupled, in different services and processes. And as all active things in CAP are services, so are usually emitters and receivers of events. Typical patterns look like that: ```js class Emitter extends cds.Service { async someMethod() { // inform unknown receivers about something happened await this.emit ('some event', { some:'payload' }) }} ``` ```js class Receiver extends cds.Service { async init() { // connect to and register for events from Emitter const Emitter = await cds.connect.to('Emitter') Emitter.on ('some event', msg => {...}) }} ``` ::: tip Emitters vs Receivers **Emitters** usually emit messages to *themselves* to inform *potential* listeners about certain events. **Receivers** connect to *Emitters* to register handlers to such emitted events. ::: ### Ubiquitous Notion of Events A *Request* in CAP is actually a specialization of an *Event Message*. The same intrinsic mechanisms of sending and reacting to events are used for asynchronous communication in inverse order. A typical flow: ![Clients send requests to services which are handled in event handlers.](assets/sync.drawio.svg) Asynchronous communication looks similar, just with reversed roles: ![Services emit event. Receivers subscribe to events which are handled in event hanlders. ](assets/async.drawio.svg) ::: tip Event Listeners vs Interceptors Requests are handled the same ways as events, with one major difference: While `on` handlers for events are *listeners* (all are called), handlers for synchronous requests are *interceptors* (only the topmost is called by the framework). An interceptor then decides whether to pass down control to `next` handlers or not. ::: ### Asynchronous & Synchronous APIs To sum up, handling events in CAP is done in the same way as you would handle requests in a service provider. Also, emitting event messages is similar to sending requests. The major difference is that the initiative is inverted: While *Consumers* connect to *Services* in synchronous communications, the *Receivers* connect to _Emitters_ in asynchronous ones; _Emitters_ in turn don't know _Receivers_. ![This graphic is explained in the accompanying text.](assets/sync-async.drawio.svg) ::: tip Blurring the line between synchronous and asynchronous API In essence, services receive events. The emitting service itself or other services can register handlers for those events in order to implement the logic of how to react to these events. ::: ### Why Using Messaging? Using messaging has two major advantages: ::: tip Resilience If a receiving service goes offline for a while, event messages are safely stored, and guaranteed to be delivered to the receiver as soon as it goes online again. ::: ::: tip Decoupling Emitters of event messages are decoupled from the receivers and don't need to know them at the time of sending. This way a service is able to emit events that other services can register on in the future, for example, to implement **extension** points. ::: ## Books Reviews Sample The following explanations walk us through a books review example from cap/samples: * **[@capire/bookshop](https://github.com/sap-samples/cloud-cap-samples/tree/main/bookshop)** provides the well-known basic bookshop app. * **[@capire/reviews](https://github.com/sap-samples/cloud-cap-samples/tree/main/reviews)** provides an independent service to manage reviews. * **[@capire/bookstore](https://github.com/sap-samples/cloud-cap-samples/tree/main/bookstore)** combines both into a composite application. ![This graphic is explained in the accompanying text.](assets/cap-samples.drawio.svg) ::: tip Follow the instructions in [*cap/samples/readme*](https://github.com/SAP-samples/cloud-cap-samples#readme) for getting the samples and exercising the following steps. ::: ### Declaring Events in CDS Package `@capire/reviews` essentially provides a `ReviewsService`, [declared like that](https://github.com/sap-samples/cloud-cap-samples/blob/main/reviews/srv/reviews-service.cds): ```cds service ReviewsService { // Sync API entity Reviews as projection on my.Reviews excluding { likes } action like (review: Reviews:ID); action unlike (review: Reviews:ID); // Async API event reviewed : { subject : Reviews:subject; count : Integer; rating : Decimal; // new avg rating } } ``` [Learn more about declaring events in CDS.](../../cds/cdl#events){.learn-more} As you can read from the definitions, the service's synchronous API allows to create, read, update, and delete user `Reviews` for arbitrary review subjects. In addition, the service's asynchronous API declares the `reviewed` event that shall be emitted whenever a subject's average rating changes. ::: tip **Services in CAP** combine **synchronous** *and* **asynchronous** APIs. Events are declared on conceptual level focusing on domain, instead of low-level wire protocols. ::: ### Emitting Events Find the code to emit events in *[@capire/reviews/srv/reviews-service.js](https://github.com/SAP-samples/cloud-cap-samples/blob/139d9574950d1a5ead475c7b47deb174418500e4/reviews/srv/reviews-service.js#L12-L20)*: ```js class ReviewsService extends cds.ApplicationService { async init() { // Emit a `reviewed` event whenever a subject's avg rating changes this.after (['CREATE','UPDATE','DELETE'], 'Reviews', (req) => { let { subject } = req.data, count, rating //= ... return this.emit ('reviewed', { subject, count, rating }) }) }} ``` [Learn more about `srv.emit()` in Node.js.](../../node.js/core-services#srv-emit-event){.learn-more} [Learn more about `srv.emit()` in Java.](../../java/services#an-event-based-api){.learn-more} Method `srv.emit()` is used to emit event messages. As you can see, emitters usually emit messages to themselves, that is, `this`, to inform potential listeners about certain events. Emitters don't know the receivers of the events they emit. There might be none, there might be local ones in the same process, or remote ones in separate processes. ::: tip Messaging on Conceptual Level Simply use `srv.emit()` to emit events, and let the CAP framework care for wire protocols like CloudEvents, transports via message brokers, multitenancy handling, and so forth. ::: ### Receiving Events Find the code to receive events in *[@capire/bookstore/srv/mashup.js](https://github.com/SAP-samples/cloud-cap-samples/blob/30764b261b6bf95854df59f54a8818a4ceedd462/bookstore/srv/mashup.js#L39-L47)* (which is the basic bookshop app enhanced by reviews, hence integration with `ReviewsService`): ```js // Update Books' average ratings when reviews are updated ReviewsService.on ('reviewed', (msg) => { const { subject, count, rating } = msg.data // ... }) ``` [Learn more about registering event handlers in Node.js.](../../node.js/core-services#srv-on-before-after){.learn-more} [Learn more about registering event handlers in Java.](../../java/event-handlers/#introduction-to-event-handlers){.learn-more} The message payload is in the `data` property of the inbound `msg` object. ::: tip To have more control over imported service definitions, you can set the `model` configuration of your external service to a cds file where you define the external service and only use the imported definitions your app needs. This way, plugins like [Open Resource Discovery (ORD)](../../plugins/#ord-open-resource-discovery) know which parts of the external service you actually use in your application. ::: ## In-Process Eventing As emitting and handling events is an intrinsic feature of the CAP core runtimes, there's nothing else required when emitters and receivers live in the same process. ![This graphic is explained in the accompanying text.](assets/local.drawio.svg) Let's see that in action... ### 1. Start a Single Server Process {#start-single-server} Run the following command to start a reviews-enhanced bookshop as an all-in-one server process: ```sh cds watch bookstore ``` It produces a trace output like that: ```log [cds] - mocking ReviewsService { path: '/reviews', impl: '../reviews/srv/reviews-service.js' } [cds] - mocking OrdersService { path: '/orders', impl: '../orders/srv/orders-service.js' } [cds] - serving CatalogService { path: '/browse', impl: '../bookshop/srv/cat-service.js' } [cds] - serving AdminService { path: '/admin', impl: '../bookshop/srv/admin-service.js' } [cds] - server listening on { url: 'http://localhost:4004' } [cds] - launched at 5/25/2023, 4:53:46 PM, version: 7.0.0, in: 991.573ms ``` As apparent from the output, both, the two bookshop services `CatalogService` and `AdminService` as well as our new `ReviewsService`, are served in the same process (mocked, as the `ReviewsService` is configured as required service in _bookstore/package.json_). ### 2. Add or Update Reviews {#add-or-update-reviews} Now, open [http://localhost:4004/reviews](http://localhost:4004/reviews) to display the Vue.js UI that is provided with the reviews service sample: ![A vue.js UI, showing the bookshop sample with the adding a review functionality](assets/capire-reviews.png) - Choose one of the reviews. - Change the 5-star rating with the dropdown. - Choose *Submit*. - Enter *bob* to authenticate. → In the terminal window you should see a server reaction like this: ```log [cds] - PATCH /reviews/Reviews/148ddf2b-c16a-4d52-b8aa-7d581460b431 < emitting: reviewed { subject: '201', count: 2, rating: 4.5 } > received: reviewed { subject: '201', count: 2, rating: 4.5 } ``` Which means the `ReviewsService` emitted a `reviewed` message that was received by the enhanced `CatalogService`. ### 3. Check Ratings in Bookshop App Open [http://localhost:4004/bookshop](http://localhost:4004/bookshop) to see the list of books served by `CatalogService` and refresh to see the updated average rating and reviews count: ![A vue.js UI showing the pure bookhsop sample without additional features.](assets/capire-books.png) ## Using Message Channels When emitters and receivers live in separate processes, you need to add a message channel to forward event messages. CAP provides messaging services, which take care for that message channel behind the scenes as illustrated in the following graphic: ![The reviews service and the catalog service, each in a seperate process, are connected to the messaging service which holds the messaging channel behind the scenes.](assets/remote.drawio.svg) ::: tip Uniform, Agnostic Messaging CAP provides messaging services, which transport messages behind the scenes using different messaging channels and brokers. All of this happens without the need to touch your code, which stays on conceptual level. ::: ### 1. Use `file-based-messaging` in Development For quick tests during development, CAP provides a simple file-based messaging service implementation. Configure that as follows for the `[development]` profile: ```jsonc "cds": { "requires": { "messaging": { "[development]": { "kind": "file-based-messaging" } }, } } ``` [Learn more about `cds.env` profiles.](../../node.js/cds-env#profiles){.learn-more} In our samples, you find that in [@capire/reviews/package.json](https://github.com/SAP-samples/cloud-cap-samples/blob/main/reviews/package.json) as well as [@capire/bookstore/package.json](https://github.com/SAP-samples/cloud-cap-samples/blob/main/bookstore/package.json), which you'll run in the next step as separate processes. ### 2. Start the `reviews` Service and `bookstore` Separately First start the `reviews` service separately: ```sh cds watch reviews ``` The trace output should contain these lines, confirming that you're using `file-based-messaging`, and that the `ReviewsService` is served by that process at port 4005: ```log [cds] - connect to messaging > file-based-messaging { file: '~/.cds-msg-box' } [cds] - serving ReviewsService { path: '/reviews', impl: '../reviews/srv/reviews-service.js' } [cds] - server listening on { url: 'http://localhost:4005' } [cds] - launched at 5/25/2023, 4:53:46 PM, version: 7.0.0, in: 593.274ms ``` Then, in a separate terminal start the `bookstore` server as before: ```sh cds watch bookstore ``` This time the trace output is different to [when you started all in a single server](#start-single-server). The output confirms that you're using `file-based-messaging`, and that you now *connected* to the separately started `ReviewsService` at port 4005: ```log [cds] - connect to messaging > file-based-messaging { file: '~/.cds-msg-box' } [cds] - mocking OrdersService { path: '/orders', impl: '../orders/srv/orders-service.js' } [cds] - serving CatalogService { path: '/browse', impl: '../reviews/srv/cat-service.js' } [cds] - serving AdminService { path: '/admin', impl: '../reviews/srv/admin-service.js' } [cds] - connect to ReviewsService > odata { url: 'http://localhost:4005/reviews' } [cds] - server listening on { url: 'http://localhost:4004' } [cds] - launched at 5/25/2023, 4:55:46 PM, version: 7.0.0, in: 1.053s ``` ### 3. Add or Update Reviews {#add-or-update-reviews-2} Similar to before, open [http://localhost:4005/vue/index.html](http://localhost:4005/vue/index.html) to add or update reviews. → In the terminal window for the `reviews` server you should see this: ```log [cds] - PATCH /reviews/Reviews/74191a20-f197-4829-bd47-c4676710e04a < emitting: reviewed { subject: '251', count: 1, rating: 3 } ``` → In the terminal window for the `bookstore` server you should see this: ```log > received: reviewed { subject: '251', count: 1, rating: 3 } ``` ::: tip **Agnostic Messaging APIs** Without touching any code the event emitted from the `ReviewsService` got transported via `file-based-messaging` channel behind the scenes and was received in the `bookstore` as before, when you used in-process eventing → which was to be shown (*QED*). ::: ### 4. Shut Down and Restart Receiver → Resilience by Design You can simulate a server outage to demonstrate the value of messaging for resilience as follows: 1. Terminate the `bookstore` server with Ctrl + C in the respective terminal. 2. Add or update more reviews as described before. 3. Restart the receiver with `cds watch bookstore`. → You should see some trace output like that: ```log [cds] - server listening on { url: 'http://localhost:4004' } [cds] - launched at 5/25/2023, 10:45:42 PM, version: 7.0.0, in: 1.023s [cds] - [ terminate with ^C ] > received: reviewed { subject: '207', count: 1, rating: 2 } > received: reviewed { subject: '207', count: 1, rating: 2 } > received: reviewed { subject: '207', count: 1, rating: 2 } ``` ::: tip **Resilience by Design** All messages emitted while the receiver was down stayed in the messaging queue and are delivered when the server is back. ::: ### Have a Look Into _~/.cds-msg-box_ You can watch the messages flowing through the message queue by opening _~/.cds-msg-box_ in a text editor. When the receiver is down and therefore the message not already consumed, you can see the event messages emitted by the `ReviewsService` in entries like that: ```json ReviewsService.reviewed {"data":{"subject":"201","count":4,"rating":5}, "headers": {...}} ``` ## Using Multiple Channels By default CAP uses a single message channel for all messages. For example: If you consume messages from SAP S/4HANA in an enhanced version of `bookstore`, as well as emit messages a customer could subscribe and react to in a customer extension, the overall topology would look like that: ![The reviews service, bookstore, and the SAP S/4HANA system send events to a common message bus. The bookstore also receives events and customer extensions as well.](assets/composite1.drawio.svg) ### Using Separate Channels Now, sometimes you want to use separate channels for different emitters or receivers. Let's assume you want to have a dedicated channel for all events from SAP S/4HANA, and yet another separate one for all outgoing events, to which customer extensions can subscribe too. This situation is illustrated in this graphic: ![The graphic shows seperate message channels for each event emitter and its subscribers.](assets/composite2.drawio.svg) This is possible when using [low-level messaging](#low-level-messaging), but comes at the price of loosing all advantages of conceptual-level messaging as explained in the following. ### Using `composite-messaging` Implementation To avoid falling back to low-level messaging, CAP provides the `composite-messaging` implementation, which basically acts like a transparent dispatcher for both, inbound and outbound messages. The resulting topology would look like that: ![Each emitter and subscriber has its own message channel. In additon there's a composite message channel that dispatches to/from each of those seperate channels.](assets/composite3.drawio.svg) ::: tip **Transparent Topologies** The `composite-messaging` implementation allows to flexibly change topologies of message channels at deployment time, without touching source code or models. ::: ### Configuring Individual Channels and Routes You would configure this in `bookstore`'s _package.json_ as follows: ```jsonc "cds": { "requires": { "messaging": { "kind": "composite-messaging", "routes": { "ChannelA": ["**/ReviewsService/*"], "ChannelB": ["**/sap/s4/**"] "ChannelC": ["**/bookshop/**"] } }, "ChannelA": { "kind": "enterprise-messaging", ... }, "ChannelB": { "kind": "enterprise-messaging", ... }, "ChannelC": { "kind": "enterprise-messaging", ... } } } ``` In essence, you first configure a messaging service for each channel. In addition, you would configure the default `messaging` service to be of kind `composite-messaging`. In the `routes`, you can use the glob pattern to define filters for event names, that means: - `**` will match any number of characters. - `*` will match any number of characters except `/` and `.`. - `?` will match a single character. ::: tip You can also refer to events declared in CDS models, by using their fully qualified event name (unless annotation `@topic` is used on them). ::: ## Low-Level Messaging In the previous sections it's documented how CAP promotes messaging on conceptual levels, staying agnostic to topologies and message brokers. While CAP strongly recommends staying on that level, CAP also offers lower-level messaging, which loses some of the advantages but still stays independent from specific message brokers. ::: tip Messaging as Just Another CAP Service All messaging implementations are provided through class `cds.MessagingService` and broker-specific subclasses of that. This class is in turn a standard CAP service, derived from `cds.Service`, hence it's consumed as any other CAP service, and can also be extended by adding event handlers as usual. ::: #### Configure Messaging Services As with all other CAP services, add an entry to `cds.requires` in your _package.json_ or _.cdsrc.json_ like that: ```jsonc "cds": { "requires": { "messaging": { "kind": // ... }, } } ``` [Learn more about `cds.env` and `cds.requires`.](../../node.js/cds-env#services){.learn-more} You're free how you name your messaging service. Could be `messaging` as in the previous example, or any other name you choose. You can also configure multiple messages services with different names. #### Connect to the Messaging Service Instead of connecting to an emitter service, connect to the messaging service: ```js const messaging = await cds.connect.to('messaging') ``` #### Emit Events to Messaging Service Instead of emitter services emitting to themselves, emit to the messaging service: ```js await messaging.emit ('ReviewsService.reviewed', { ... }) ``` #### Receive Events from Messaging Service Instead of registering event handlers with a concrete emitter service, register handlers on the messaging service: ```js messaging.on ('ReviewsService.reviewed', msg => console.log(msg)) ```
#### Declared Events and `@topic` Names When declaring events in CDS models, be aware that the fully qualified name of the event is used as topic names when emitting to message brokers. Based on the following model, the resulting topic name is `my.namespace.SomeEventEmitter.SomeEvent`. ```cds namespace my.namespace; service SomeEventEmitter { event SomeEvent { ... } } ``` If you want to manually define the topic, you can use the `@topic` annotation: ```cds //... @topic: 'some.very.different.topic-name' event SomeEvent { ... } ``` #### Conceptual vs. Low-Level Messaging When looking at the previous code samples, you see that in contrast to conceptual messaging you need to provide fully qualified event names now. This is just one of the advantages you lose. Have a look at the following list of advantages you have using conceptual messaging and lose with low-level messaging. - Service-local event names (as already mentioned) - Event declarations (as they go with individual services) - Generated typed API classes for declared events - Run in-process without any messaging service ::: tip Always prefer conceptual-level API over low-level API variants. Besides the things listed above, this allows you to flexibly change topologies, such as starting with co-located services in a single process, and moving single services out to separate micro services later on. ::: ## CloudEvents Standard {#cloudevents} CAP messaging has built-in support for formatting event data compliant to the [CloudEvents](https://cloudevents.io/) standard. Enable this using the `format` config option as follows: ```json "cds": { "requires": { "messaging": { "format": "cloudevents" } } } ``` With this setting, all mandatory and some more basic header fields, like `type`, `source`, `id`, `datacontenttype`, `specversion`, `time` are filled in automatically. The event name is used as `type`. The message payload is in the `data` property anyways. ::: tip CloudEvents is a wire protocol specification. Application developers shouldn't have to care for such technical details. CAP ensures that for you, by filling in the respective fields behind the scenes. ::: ## [Using SAP Event Mesh](./event-mesh) {#sap-event-mesh} CAP has out-of-the-box support for SAP Event Mesh. As an application developer, all you need to do is configuring CAP to use `enterprise-messaging`, usually in combination with `cloudevents` format, as in this excerpt from a _package.json_: ```jsonc "cds": { "requires": { "messaging": { "[production]": { "kind": "enterprise-messaging", "format": "cloudevents" } } } } ``` [Learn more about `cds.env` profiles](../../node.js/cds-env#profiles){.learn-more} ::: tip Read the guide Find additional information about deploying SAP Event Mesh on SAP BTP in this guide: [→ **_Using SAP Event Mesh in BTP_**](./event-mesh) ::: ## [Using SAP Cloud Application Event Hub](./event-broker) {#sap-event-broker} CAP has growing out-of-the-box support for SAP Cloud Application Event Hub. As an application developer, all you need to do is configuring CAP to use `event-broker`, as in this excerpt from a _package.json_: ```jsonc "cds": { "requires": { "messaging": { "[production]": { "kind": "event-broker" } } } } ``` [Learn more about `cds.env` profiles](../../node.js/cds-env#profiles){.learn-more} ::: tip Read the guide Find additional information about deploying SAP Event Broper on SAP BTP in this guide: [→ **_Using SAP Cloud Application Event Hub in BTP_**](./event-broker) ::: ## [Events from SAP S/4HANA](./s4) SAP S/4HANA integrates SAP Event Mesh as well as SAP Cloud Application Event Hub for messaging. That makes it relatively easy for CAP-based applications to receive events from SAP S/4HANA systems. In contrast to CAP, the asynchronous APIs of SAP S/4HANA are separate from the synchronous ones (OData, REST). So, the effort on the CAP side is to fill this gap. You can achieve it like that, for example, for an already imported SAP S/4HANA BusinessPartner API: ```cds // filling in missing events as found on SAP Business Accelerator Hub using { API_BUSINESS_PARTNER as S4 } from './API_BUSINESS_PARTNER'; extend service S4 with { event BusinessPartner.Created @(topic:'sap.s4.beh.businesspartner.v1.BusinessPartner.Created.v1') { BusinessPartner : String } event BusinessPartner.Changed @(topic:'sap.s4.beh.businesspartner.v1.BusinessPartner.Changed.v1') { BusinessPartner : String } } ``` [Learn more about importing SAP S/4HANA service APIs.](../using-services#external-service-api){.learn-more} With that gap filled, we can easily receive events from SAP S/4HANA the same way as from CAP services as explained in this guide, for example: ```js const S4Bupa = await cds.connect.to ('API_BUSINESS_PARTNER') S4Bupa.on ('BusinessPartner.Changed', msg => {...}) ``` ::: tip Read the guide Find more detailed information specific to receiving events from SAP S/4HANA in this separate guide: [→ **_Receiving Events from SAP S/4HANA_**](./s4) ::: # Using SAP Event Mesh in Cloud Foundry CAP provides out-of-the-box support for [SAP Event Mesh](https://help.sap.com/docs/event-mesh), and automatically handles many things behind the scenes, so that application coding stays agnostic and focused on conceptual messaging. ::: warning The following guide is based on a productive (paid) account on SAP BTP. It's not supported to use the trial offering of SAP Event Mesh. ::: ## Prerequisite: Create an Instance of SAP Event Mesh - [Follow this tutorial](https://developers.sap.com/group.cp-enterprisemessaging-get-started.html) to create an instance of SAP Event Mesh with plan `default`. - Alternatively follow [one of the guides in SAP Help Portal](https://help.sap.com/docs/SAP_EM/bf82e6b26456494cbdd197057c09979f/3ef34ffcbbe94d3e8fff0f9ea2d5911d.html). ::: tip **Important:** You don't need to manually create queues or queue subscriptions as CAP takes care for that automatically based on declared events and subscriptions. ::: ## Use `enterprise-messaging` Add the following to your _package.json_ to use SAP Event Mesh: ```jsonc "cds": { "requires": { "messaging": { "[production]": { "kind": "enterprise-messaging" }, } } } ``` [Learn more about `cds.env` profiles](../../node.js/cds-env#profiles){.learn-more}

**Behind the Scenes**, the `enterprise-messaging` implementation handles these things automatically and transparently: - Creation of queues & subscriptions for event receivers - Handling all broker-specific handshaking and acknowledgments - Constructing topic names as expected by the broker - Wire protocol envelopes, that is, CloudEvents

### Optional: Add `namespace` Prefixing Rules SAP Event Mesh documentation recommends to prefix all event names with the service instance's configured `namespace`, both, when emitting as well as when subscribing to events. If you followed these rules, add corresponding rules to your configuration in _package.json_ to have CAP's messaging service implementations enforcing these rules automatically: ```json "cds": { "requires": { "messaging": { "publishPrefix": "$namespace/", "subscribePrefix": "$namespace/" } } } ``` The variable `$namespace` is resolved from your SAP Event Mesh service instance's configured `namespace` property. ## Run Tests in `hybrid` Setup Before [deploying to the cloud](#deploy-to-the-cloud-with-mta), you may want to run some ad-hoc tests with a hybrid setup, that is, keep running the CAP services locally, but using the SAP Event Mesh instance from the cloud. Do that as follows: 1. Configure CAP to use the `enterprise-messaging-shared` implementation in the `reviews` and `bookstore` sample: ```jsonc "cds": { "requires": { "messaging": { "[hybrid]": { "kind": "enterprise-messaging-shared" } } } } ``` > The `enterprise-messaging-shared` variant is for single-tenant usage and uses AMQP by default. Thus, it requires much less setup for local tests compared to the production variant, which uses HTTP-based protocols by default. 2. Add `@sap/xb-msg-amqp-v100` as dependency to `reviews` and `bookstore`: ```sh npm add @sap/xb-msg-amqp-v100 ``` [Learn more about SAP Event Mesh (Shared).](../../node.js/messaging#event-mesh-shared){.learn-more} 3. Create a service key for your Event Mesh instance [→ see help.sap.com](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/4514a14ab6424d9f84f1b8650df609ce.html) 4. Bind to your Event Mesh instance's service key from `reviews` and `bookstore`: ```sh cds bind -2 : ``` [Learn more about `cds bind` and hybrid testing.](../../advanced/hybrid-testing){.learn-more} 5. Run your services in separate terminal shells with the `hybrid` profile: ```sh cds watch reviews --profile hybrid ``` ```sh cds watch bookstore --profile hybrid ``` [Learn more about `cds.env` profiles.](../../node.js/cds-env#profiles){.learn-more} 6. Test your app [as described in the messaging guide](./#add-or-update-reviews). ### CAP Automatically Creates Queues and Subscriptions When you run the services with a bound instance of SAP Event Mesh as documented in a previous section, CAP messaging service implementations will automatically create a queue for each receiver process. The queue name is chosen automatically and the receiver's subscriptions added. ### Optional: Configure Queue Names In case you want to manage queues yourself, use config option `queue.name` as follows: ```jsonc "cds": { "requires": { "messaging": { // ... "queue": { "name": "$namespace/my/own/queue" } } } } ``` In both cases — automatically chosen queue names or explicitly configured ones — if the queue already exists it's reused, otherwise it's created. [Learn more about queue configuration options.](../../node.js/messaging#message-brokers){.learn-more} ## Deploy to the Cloud (with MTA) A general description of how to deploy CAP applications to SAP BTP's Cloud Foundry, can be found in the [Deploy to Cloud* guide](../deployment/). As documented there, MTA is frequently used to deploy to SAP BTP. Follow these steps to ensure binding of your deployed application to the SAP Event Mesh instance. ### 1. Specify Binding to SAP Event Mesh Instance Add SAP Event Mesh's service instance's name to the `requires` section of your CAP application's module, and a matching entry to the `resources` section, for example: ```yaml modules: - name: bookstore-srv requires: - name: resources: # SAP Event Mesh - name: type: org.cloudfoundry.managed-service parameters: service: enterprise-messaging service-plan: ``` [Learn more about using MTA.](../deployment/){.learn-more} ::: warning Make sure to use the exact `name` and `service-plan` used at the time creating the service instance you want to use. ::: ### 2. Optional: Auto-Create SAP Event Mesh Instances MTA can also create the service instance automatically. To do so, you need to additionally provide a service descriptor file and reference that through the `path` parameter in the `resources` section: ```yaml resources: # SAP Event Mesh as above... parameters: path: ./ ``` [Learn more about Service Descriptors for SAP Event Mesh.](https://help.sap.com/docs/SAP_EM/bf82e6b26456494cbdd197057c09979f/5696828fd5724aa5b26412db09163530.html){.learn-more} # Using SAP Cloud Application Event Hub in Cloud Foundry [SAP Cloud Application Event Hub](https://help.sap.com/docs/event-broker) is the new default offering for messaging in SAP Business Technology Platform (SAP BTP). CAP provides out-of-the-box support for SAP Cloud Application Event Hub, and automatically handles many things behind the scenes, so that application coding stays agnostic and focused on conceptual messaging. ::: warning The following guide is based on a productive (paid) account on SAP BTP. ::: ## Consuming Events in a Stand-alone App { #consume-standalone } This guide describes the end-to-end process of developing a stand-alone (or "single tenant") CAP application that consumes messages via SAP Cloud Application Event Hub. The guide uses SAP S/4HANA as the event emitter, but this is a stand-in for any system that is able to publish CloudEvents via SAP Cloud Application Event Hub. Sample app: [@capire/incidents with Customers based on S/4's Business Partners](https://github.com/cap-js/incidents-app/tree/event-broker) ### Prerequisite: Events & Messaging in CAP From the perspective of a CAP developer, SAP Cloud Application Event Hub is yet another messaging broker. That is to say, CAP developers focus on [modeling their domain](../domain-modeling) and [implementing their domain-specific custom logic](../providing-services#custom-logic). Differences between the various event transporting technologies are held as transparent as possible. Hence, before diving into this guide, you should be familiar with the general guide for [Events & Messaging in CAP](../messaging/), as it already covers the majority of the content. Since SAP Cloud Application Event Hub is based on the [CloudEvents](https://cloudevents.io/) standard, the `@topic` annotation for events in your CDS model is interpreted as the CloudEvents `type` attribute. ### Add Events and Handlers There are two options for adding the events that shall be consumed to your model, and subsequently registering event handlers for the same. #### 1. Import and Augment This approach is described in [Events from SAP S/4HANA](../messaging/#events-from-sap-s-4hana), [Receiving Events from SAP S/4HANA Cloud Systems](../messaging/s4), and specifically [Consume Events Agnostically](../messaging/s4#consume-events-agnostically) regarding handler registration. #### 2. Using Low-Level Messaging As a second option, you can skip the modeling part and simply use [Low-Level Messaging](../messaging/s4#using-low-level-messaging). However, please note that future [Open Resource Discovery (ORD)](https://sap.github.io/open-resource-discovery/) integration will most likely benefit from modeled approaches. ### Use `event-broker` Configure your application to use the messaging service kind `event-broker` (derived from SAP Cloud Application Event Hub's technical name). [Learn more about configuring SAP Cloud Application Event Hub in CAP Node.js](../../node.js/messaging#event-broker){.learn-more} [Learn more about `cds.env` profiles](../../node.js/cds-env#profiles){.learn-more} ::: tip Local Testing Since SAP Cloud Application Event Hub sends events via HTTP, you won't be able to receive events on your local machine unless you use a tunneling service. Therefore we recommend to use a messaging service of kind [`local-messaging`](../../node.js/messaging#local-messaging) for local testing. ::: ### Deploy to the Cloud (with MTA) Please see [Deploy to Cloud Foundry](../deployment/to-cf) regarding deployment with MTA as well as the deployment section from [SAP Cloud Application Event Hub in CAP Node.js](../../node.js/messaging#event-broker). ### Connecting it All Together In SAP BTP System Landscape, add a new system of type `SAP BTP Application` for your CAP application including its integration dependencies, connect all involved systems (incl. SAP Cloud Application Event Hub) into a formation and enable the event subscription. For more details, please refer to guide [CAP Application as a Consumer](https://help.sap.com/docs/event-broker/event-broker-service-guide/cap-application-as-subscriber) in the official documentation of SAP Cloud Application Event Hub. ::: tip Test Events For testing purposes, SAP S/4HANA can send technical test events of type `sap.eee.iwxbe.testproducer.v1.Event.Created.v1` which your app can subscribe to. You can trigger such events with _Enterprise Event Enablement - Event Monitor_. ::: # Receiving Events from SAP S/4HANA Cloud Systems SAP S/4HANA integrates SAP Event Mesh as well as SAP Cloud Application Event Hub for messaging. Hence, it is relatively easy for CAP-based application to receive events from SAP S/4HANA systems. This guide provides detailed information on that. ## Find & Import APIs As documented in the [Service Consumption guide](../using-services#external-service-api), get, and `cds import` the API specification of an SAP S/4HANA service you want to receive events from. For example, for "BusinessPartner" using [SAP Business Accelerator Hub](https://api.sap.com/): 1. Find / open [Business Partner (A2X) API](https://api.sap.com/api/API_BUSINESS_PARTNER). 2. Choose button *"API Specification"*. 3. Download the EDMX spec from this list: ![Showing all available specifications on the SAP Business Accelerator Hub.](./assets/api-specification.png){ } 1. Import it as a CDS model: ```sh cds import ``` [Learn more about importing SAP S/4HANA service APIs.](../using-services#external-service-api){.learn-more} ## Find Information About Events For example, using [SAP Business Accelerator Hub](https://api.sap.com/): 1. [Find the BusinessPartner Events page.](https://api.sap.com/event/SAPS4HANABusinessEvents_BusinessPartnerEvents/overview) 2. Choose _View Event Reference_. 3. Expand the _POST_ request shown. 4. Choose _Schema_ tab. 5. Expand the `data` property. ![Shows the event reference page on the SAP Business Accelerator Hub, highlighting the data property.](assets/business-partner-events.png){.mute-dark} The expanded part, highlighted in red, tells you all you need to know: - the event name: `sap.s4.beh.businesspartner.v1.BusinessPartner.Changed.v1` - the payload's schema → in `{...}` > All the other information on this page can be ignored, as it's about standard CloudEvents wire format attributes, which are always the same, and handled automatically by CAP behind the scenes for you. ## Add Missing Event Declarations In contrast to CAP, the asynchronous APIs of SAP S/4HANA are separate from synchronous APIs (that is, OData, REST). On CAP side, you need to fill this gap. For example, for an already imported SAP S/4HANA BusinessPartner API: ```cds // filling in missing events as found on SAP Business Accelerator Hub using { API_BUSINESS_PARTNER as S4 } from './API_BUSINESS_PARTNER'; extend service S4 with { event BusinessPartner.Created @(topic:'sap.s4.beh.businesspartner.v1.BusinessPartner.Created.v1') { BusinessPartner : String } event BusinessPartner.Changed @(topic:'sap.s4.beh.businesspartner.v1.BusinessPartner.Changed.v1') { BusinessPartner : String } } ``` ::: tip If using SAP Event Mesh, please see [CloudEvents Standard](./index.md#cloudevents) and [Node - Messaging - CloudEvents Protocol](../../node.js/messaging.md#cloudevents-protocol) to learn about `format: 'cloudevents'`, `publishPrefix` and `subscribePrefix`. :::

## Consume Events Agnostically With agnostic consumption, you can easily receive events from SAP S/4HANA the same way as from CAP services as already explained in this guide, for example like that: ```js const S4Bupa = await cds.connect.to ('API_BUSINESS_PARTNER') S4bupa.on ('BusinessPartner.Changed', msg => {...}) ``` ## Configure CAP To ease the pain of the afore-mentioned topic rewriting effects, CAP has built-in support for [SAP Event Mesh](./event-mesh) as well as [SAP Cloud Application Event Hub](./event-broker). Configure the messaging service as follows, to let it automatically create correct technical topics to subscribe to SAP S/4HANA events: For SAP Event Mesh: ```json "cds": { "requires": { "messaging": { "kind": "enterprise-messaging-shared", "format": "cloudevents", // implicitly applied default prefixes "publishPrefix": "$namespace/ce/", "subscribePrefix": "+/+/+/ce/" } } } ``` **Note:** In contrast to the default configuration recommended in the [SAP Event Mesh documentation](https://help.sap.com/docs/SAP_EM/bf82e6b26456494cbdd197057c09979f/5499e2e74e674c69b057072272c80d4f.html), ensure you configure your service instance to allow the pattern `+/+/+/ce/*` for subscriptions. That is, **do not** restrict `subscribeFilter`s to `${namespace}`! For SAP Cloud Application Event Hub: ```json "cds": { "requires": { "messaging": { "kind": "event-broker" } } } ``` With that, your developers can enter event names as they're found on SAP Business Accelerator Hub. And our CDS extensions, as previously described, simplify to that definition: ```cds // filling in missing events as found on SAP Business Accelerator Hub using { API_BUSINESS_PARTNER as S4 } from './API_BUSINESS_PARTNER'; extend service S4 with { event BusinessPartner.Created @(topic:'sap.s4.beh.businesspartner.v1.BusinessPartner.Changed.v1') { BusinessPartner : String } event BusinessPartner.Changed @(topic:'sap.s4.beh.businesspartner.v1.BusinessPartner.Created.v1') { BusinessPartner : String } } ``` ## Configure SAP S/4HANA As a prerequisite for consuming SAP S/4HANA events, the SAP S/4HANA system itself needs to be configured to send out specific event messages to a specific SAP Event Mesh or SAP Cloud Application Event Hub service instance. How to create the necessary service instances and use them with a CAP application was already described in the previous sections [SAP Event Mesh](./event-mesh) and [SAP Cloud Application Event Hub](./event-broker), respectively. A description of how to configure an SAP S/4HANA system to send out specific events is out of scope of this documentation here. See [this documentation](https://help.sap.com/docs/SAP_S4HANA_CLOUD/0f69f8fb28ac4bf48d2b57b9637e81fa/82e97d5329044732af1efd996bfdc2ab.html) for more details. ## Using Low-Level Messaging Instead of adding events found on [SAP Business Accelerator Hub](https://api.sap.com/content-type/Events/events/packages) to a CDS service model, it's also possible to use a messaging service directly to consume events from SAP S/4HANA. You have to bind the `messaging` service directly to the SAP Event Mesh or SAP Cloud Application Event Hub service instance that the SAP S/4HANA system sends the event messages to. Then you can consume the event by registering a handler on the `type` of the event that should be received (`sap.s4.beh.businesspartner.v1.BusinessPartner.Changed.v1` in the example): ```js const messaging = await cds.connect.to ('messaging') messaging.on ('sap.s4.beh.businesspartner.v1.BusinessPartner.Changed.v1', (msg) => { const { BusinessPartner } = msg.data console.log('--> Event received: BusinessPartner changed (ID="'+BusinessPartner+'")') }) ``` All the complex processes, like determining the correct technical topic to subscribe to and adding this subscription to a queue, will be done automatically in the background. # Publishing APIs How to publish APIs in different formats
# Serving OData APIs ## Feature Overview { #overview} OData is an OASIS standard, which essentially enhances plain REST with standardized system query options like `$select`, `$expand`, `$filter`, etc. Find a rough overview of the feature coverage in the following table: | Query Options | Remarks | Node.js | Java | |----------------|-------------------------------------------|------------|---------| | `$search` | Search in multiple/all text elements⁽¹⁾| | | | `$value` | Retrieves single rows/values | | | | `$top`,`$skip` | Requests paginated results | | | | `$filter` | Like SQL where clause | | | | `$select` | Like SQL select clause | | | | `$orderby` | Like SQL order by clause | | | | `$count` | Gets number of rows for paged results | | | | `$apply` | For [data aggregation](#data-aggregation) | | | | `$expand` | Deep-read associated entities | | | | [Lambda Operators](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part2-url-conventions.html#_Toc31361024) | Boolean expressions on a collection | | ⁽²⁾ | | [Parameters Aliases](https://docs.oasis-open.org/odata/odata/v4.01/os/part1-protocol/odata-v4.01-os-part1-protocol.html#sec_ParameterAliases) | Replace literal value in URL with parameter alias | | ⁽³⁾ | - ⁽¹⁾ The elements to be searched are specified with the [`@cds.search` annotation](../guides/providing-services#searching-data). - ⁽²⁾ The navigation path identifying the collection can only contain one segment. - ⁽³⁾ Supported for key values and for parameters of functions only. System query options can also be applied to an [expanded navigation property](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part2-url-conventions.html#_Toc31361039) (nested within `$expand`): | Query Options | Remarks | Node.js | Java | |----------------|-------------------------------------------|----------|--------| | `$select` | Select properties of associated entities | | | | `$filter` | Filter associated entities | | | | `$expand` | Nested expand | | | | `$orderby` | Sort associated entities | | | | `$top`,`$skip` | Paginate associated entities | | | | `$count` | Count associated entities | | | | `$search` | Search associated entities | | | [Learn more in the **Getting Started guide on odata.org**.](https://www.odata.org/getting-started/){.learn-more} [Learn more in the tutorials **Take a Deep Dive into OData**.](https://developers.sap.com/mission.scp-3-odata.html){.learn-more} | Data Modification | Remarks | Node.js | Java | |-------------------|-------------------------------------------|------------|---------| | [Create an Entity](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_CreateanEntity) | `POST` request on Entity collection | | | | [Update an Entity](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_UpdateanEntity) | `PATCH` or `PUT` request on Entity | | | [ETags](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_UseofETagsforAvoidingUpdateConflicts) | For avoiding update conflicts | | | | [Delete an Entity](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_DeleteanEntity) | `DELETE` request on Entity | | | | [Delta Payloads](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_DeltaPayloads) | For nested entity collections in [deep updates](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_UpdateRelatedEntitiesWhenUpdatinganE) | | | | [Patch Collection](#odata-patch-collection) | Update Entity collection with [delta](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_DeltaPayloads) | | | ## PATCH Entity Collection with Mass Data (Java) { #odata-patch-collection } With OData v4, you can [update a collection of entities](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_UpdateaCollectionofEntities) with a _single_ PATCH request. The resource path of the request targets the entity collection and the body of the request is given as a [delta payload](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_DeltaPayloads): ```js PATCH /CatalogService/Books Content-Type: application/json { "@context": "#$delta", "value": [ { "ID": 17, "title": "CAP - what's new in 2023", "price": 29.99, "author_ID": 999 }, { "ID": 85, "price": 9.99 }, { "ID": 42, "@removed": { "reason": "deleted" } } ] } ``` PATCH requests with delta payload are executed using batch delete and [upsert](../java/working-with-cql/query-api#bulk-upsert) statements, and are more efficient than OData [batch requests](https://docs.oasis-open.org/odata/odata/v4.01/csprd02/part1-protocol/odata-v4.01-csprd02-part1-protocol.html#sec_BatchRequests). Use PATCH on entity collections for uploading mass data using a dedicated service, which is secured using [role-based authorization](../java/security#role-based-auth). Delta updates must be explicitly enabled by annotating the entity with ```cds @Capabilities.UpdateRestrictions.DeltaUpdateSupported ``` Limitations: * Conflict detection via [ETags](../guides/providing-services#etag) is not supported. * [Draft flow](../java/fiori-drafts#bypassing-draft-flow) is bypassed, `IsActiveEntity` has to be `true`. * [Draft locks](../java/fiori-drafts#draft-lock) are ignored, active entities are updated or deleted w/o canceling drafts. * [Added and deleted links](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_IteminaDeltaPayloadResponse) are not supported. * The header `Prefer=representation` is not yet supported. * The `continue-on-error` preference is not yet supported. * The generic CAP handler support for [upsert](../java/working-with-cql/query-api#upsert) is limited, for example, audit logging is not supported. ## Mapping of CDS Types { #type-mapping} The table below lists [CDS's built-in types](../cds/types) and their mapping to the OData EDM type system. | CDS Type | OData V4 | | -------------- | --------------------------------------- | | `UUID` | _Edm.Guid_ ⁽¹⁾ | | `Boolean` | _Edm.Boolean_ | | `UInt8 ` | _Edm.Byte_ | | `Int16` | _Edm.Int16_ | | `Int32` | _Edm.Int32_ | | `Integer` | _Edm.Int32_ | | `Int64` | _Edm.Int64_ | | `Integer64` | _Edm.Int64_ | | `Decimal` | _Edm.Decimal_ | | `Double` | _Edm.Double_ | | `Date` | _Edm.Date_ | | `Time` | _Edm.TimeOfDay_ | | `DateTime` | _Edm.DateTimeOffset_ | | `Timestamp` | _Edm.DateTimeOffset_ with Precision="7" | | `String` | _Edm.String_ | | `Binary` | _Edm.Binary_ | | `LargeBinary` | _Edm.Binary_ | | `LargeString` | _Edm.String_ | | `Map` | represented as empty, open complex type | | `Vector` | not supported ⁽²⁾ | > ⁽¹⁾ Mapping can be changed with, for example, `@odata.Type='Edm.String'` > ⁽²⁾ Type `cds.Vector` must not appear in an OData service OData V2 has the following differences: | CDS Type | OData V2 | | ------------ | ----------------------------------------------- | | `Date` | _Edm.DateTime_ with `sap:display-format="Date"` | | `Time` | _Edm.Time_ | | `Map` | not supported | ### Overriding Type Mapping { #override-type-mapping} Override standard type mappings using the annotation `@odata.Type` first, and then additionally define `@odata {MaxLength, Precision, Scale, SRID}`. `@odata.Type` is effective on scalar CDS types only and the value must be a valid OData (EDM) primitive type for the specified protocol version. Unknown types and non-matching facets are silently ignored. No further value constraint checks are applied. They allow, for example, to produce additional OData EDM types which are not available in the standard type mapping. This is done during the import of external service APIs, see [Using Services](../guides/using-services#external-service-api). ```cds entity Foo { // ... @odata: { Type: 'Edm.GeometryPolygon', SRID: 0 } geoCollection : LargeBinary; }; ``` Another prominent use case is the CDS type `UUID`, which maps to `Edm.Guid` by default. However, the OData standard puts up restrictive rules for _Edm.Guid_ values - for example, only hyphenated strings are allowed - which can conflict with existing data. Therefore, you can override the default mapping as follows: ```cds entity Books { key ID : UUID @odata.Type:'Edm.String'; // ... } ``` ::: warning This annotation affects the client side facing API only. There's no automatic data modification of any kind behind the scenes, like rounding, truncation, conversion, and so on. It's your responsibility to perform all required modifications on the data stream such that the values match their type in the API. If you are not doing the required conversions, you can "cast" any scalar CDS type into any incompatible EDM type: ```cds entity Foo { // ... @odata: {Type: 'Edm.Decimal', Scale: 'floating' } str: String(17) default '17.4'; } ``` This translates into the following OData API contract: ```xml ``` The client can now rightfully expect that float numbers are transmitted but in reality the values are still strings. ::: ## OData Annotations { #annotations} The following sections explain how to add OData annotations to CDS models and how they're mapped to EDMX outputs. Only annotations defined in the vocabularies mentioned in section [Annotation Vocabularies](#vocabularies) are considered in the translation. ### Terms and Properties OData defines a strict two-fold key structure composed of `@.` and all annotations are always specified as a _Term_ with either a primitive value, a record value, or collection values. The properties themselves may, in turn, be primitives, records, or collections. #### Example ```cds @Common.Label: 'Customer' @UI.HeaderInfo: { TypeName : 'Customer', TypeNamePlural : 'Customers', Title : { Value : name } } entity Customers { /* ... */ } ``` This is represented in CSN as follows: ```jsonc {"definitions":{ "Customers":{ "kind": "entity", "@Common.Label": "Customer", "@UI.HeaderInfo.TypeName": "Customer", "@UI.HeaderInfo.TypeNamePlural": "Customers", "@UI.HeaderInfo.Title.Value": {"=": "name"}, /* ... */ } }} ``` And would render to EDMX as follows: ```xml ``` ::: tip The value for `@UI.HeaderInfo` is flattened to individual key-value pairs in CSN and 'restructured' to a record for OData exposure in EDMX. ::: For each annotated target definition in CSN, the rules for restructuring from CSN sources are: 1. Annotations with a single-identifier key are skipped (as OData annotations always have a `@Vocabulary.Term...` key signature). 2. All individual annotations with the same `@` prefix are collected. 3. If there is only one annotation without a suffix, → that one is a scalar or array value of an OData term. 4. If there are more annotations with suffix key parts → it's a record value for the OData term. ### Qualified Annotations OData foresees [qualified annotations](https://docs.oasis-open.org/odata/odata/v4.0/errata03/os/complete/part3-csdl/odata-v4.0-errata03-os-part3-csdl-complete.html#_Toc453752511), which essentially allow to specify different values for a given property. CDS syntax for annotations was extended to also allow appending OData-style qualifiers after a `#` sign to an annotation key, but always only as the last component of a key in the syntax. For example, this is supported: ```cds @Common.Label: 'Customer' @Common.Label#Legal: 'Client' @Common.Label#Healthcare: 'Patient' @Common.ValueList: { Label: 'Customers', CollectionPath:'Customers' } @Common.ValueList#Legal: { Label: 'Clients', CollectionPath:'Clients' } ``` and would render as follows in CSN: ```json { "@Common.Label": "Customer", "@Common.Label#Legal": "Clients", "@Common.Label#Healthcare": "Patients", "@Common.ValueList.Label": "Customers", "@Common.ValueList.CollectionPath": "Customers", "@Common.ValueList#Legal.Label": "Clients", "@Common.ValueList#Legal.CollectionPath": "Clients", } ``` Note that there's no interpretation and no special handling for these qualifiers in CDS. You have to write and apply them in exactly the same way as your chosen OData vocabularies specify them. ### Primitives > Note: The `@Some` annotation isn't a valid term definition. The following example illustrates the rendering of primitive values. Primitive annotation values, meaning Strings, Numbers, `true`, and `false` are mapped to corresponding OData annotations as follows: ```cds @Some.Boolean: true @Some.Integer: 1 @Some.Number: 3.14 @Some.String: 'foo' ``` ```xml ``` #### Null Value { #null-value } A `null` value can be set either as an [annotation expression](#expression-annotations) or as a [dynamic expression](#dynamic-expressions): ```cds @Some.NullXpr: (null) // annotation expression, short form @Some.NullFunc: ($Null()) // annotation expression, functional form @Some.NullDyn: { $edmJson: { $Null } } // dynamic expression ``` All three expressions result in the following rendering: ```xml ``` [Have a look at our *CAP SFLIGHT* sample, showcasing the usage of OData annotations.](https://github.com/SAP-samples/cap-sflight/blob/main/app/travel_processor/capabilities.cds){.learn-more} ### Records > Note: The `@Some` annotation isn't a valid term definition. The following example illustrates the rendering of record values. Record-like source structures are mapped to `` nodes in EDMX, with primitive types translated analogously to the above: ```cds @Some.Record: { Null: (null), Boolean: true, Integer: 1, Number: 3.14, String: 'foo' } ``` ```xml ``` If possible, the type of the record in OData is deduced from the information in the [OData Annotation Vocabularies](#vocabularies): ```cds @Common.ValueList: { CollectionPath: 'Customers' } ``` ```xml ``` Frequently, the OData record type cannot be determined unambiguously, for example if the type found in the vocabulary is abstract. Then you need to explicitly specify the type by adding a property named `$Type` in the record. For example: ```cds @UI.Facets : [{ $Type : 'UI.CollectionFacet', ID : 'Customers' }] ``` ```xml ``` There is one exception for a very prominent case: if the deduced [record type is `UI.DataFieldAbstract`](https://github.com/SAP/odata-vocabularies/blob/main/vocabularies/UI.md), the compiler by default automatically chooses `UI.DataField`: ```cds @UI.Identification: [{ Value: deliveryId }] ``` ```xml ``` To overwrite the default, use an explicit `$Type` like shown previously. [Have a look at our *CAP SFLIGHT* sample, showcasing the usage of OData annotations.](https://github.com/SAP-samples/cap-sflight/blob/a7b166b7b9b3d2adb1640b4b68c3f8a26c6961c1/app/travel_processor/value-helps.cds){.learn-more} ### Collections > Note: The `@Some` annotation isn't a valid term definition. The following example illustrates the rendering of collection values. Arrays are mapped to `` nodes in EDMX and if primitives show up as direct elements of the array, these elements are wrapped into individual primitive child nodes of the resulting collection as is. The rules for records and collections are applied recursively: ```cds @Some.Collection: [ null, true, 1, 3.14, 'foo', { $Type:'UI.DataField', Label:'Whatever', Hidden } ] ``` ```xml true 1 3.14 foo ``` ### References { #references } > Note: The `@Some` annotation isn't a valid term definition. The following example illustrates the rendering of reference values. References in CDS annotations are mapped to `Path` properties or nested `` elements, respectively: ```cds @Some.Term: My.Reference @Some.Record: { Value: My.Reference } @Some.Collection: [ My.Reference ] ``` ```xml My/Reference ``` As the compiler isn't aware of the semantics of such references, the mapping is very simplistic: each `.` in a path is replaced by a `/`. Use [expression-valued annotations](#expression-annotations) for more convenience. Use a [dynamic expression](#dynamic-expressions) if the generic mapping can't produce the desired ``: ```cds @Some.Term: {$edmJson: {$Path: '/com.sap.foo.EntityContainer/EntityName/FieldName'}} ``` ```xml /com.sap.foo.EntityContainer/EntityName/FieldName ``` ### Enumeration Values Enumeration symbols are mapped to corresponding `EnumMember` properties in OData. Here are a couple of examples of enumeration values and the annotations that are generated. The first example is for a term in the [Common vocabulary](https://github.com/SAP/odata-vocabularies/blob/main/vocabularies/Common.md): ```cds @Common.TextFormat: #html ``` ```xml ``` The second example is for a (record type) term in the [Communication vocabulary](https://github.com/SAP/odata-vocabularies/blob/main/vocabularies/Communication.md): ```cds @Communication.Contact: { gender: #F } ``` ```xml ``` ### Expressions { #expression-annotations } If the value of an OData annotation is an [expression](../cds/cdl#expressions-as-annotation-values), the OData backend provides improved handling of references and automatic mapping from CDS expression syntax to OData expression syntax. #### Flattening In contrast to [simple references](#references), the references in expression-like annotation values are correctly handled during model transformations, like other references in the model. When the CDS model is flattened for OData, the flattening is consequentially also applied to these references, and they are translated to the flat model. ::: tip Although CAP supports structured types and elements, we recommend to use them only if they bring a real benefit. In general, you should keep your models as flat as possible. ::: Example: ```cds type Price { @Measures.ISOCurrency: (currency) amount : Decimal; currency : String(3); } service S { entity Product { key id : Integer; name : String; price : Price; } } ``` Structured element `price` of `S.Product` is unfolded to flat elements `price_amount` and `price_currency`. Accordingly, the reference in the annotation is rewritten from `currency` to `price_currency`: ```xml ``` Example: ```cds service S { entity E { key id : Integer; f : Association to F; @Some.Term: (f.struc.y) val : Integer; } entity F { key id : Integer; struc { y : Integer; } } } ``` The OData backend is aware of the semantics of a path and distinguishes association path steps from structure access. The CDS path `f.struc.y` is translated to the OData path `f/struc_y`: ```xml ``` ::: warning Restrictions concerning the foreign key elements of managed associations 1. Usually an annotation assigned to a managed association is copied to the foreign key elements of the association. This is a workaround for the lack of possibility to directly annotate a foreign key element. This copy mechanism is _not_ applied for annotations with expression values. So it is currently not possible to use expression-valued annotations for annotating foreign keys of a managed association. 2. In an expression-valued annotation, it is not possible to reference the foreign key element of a managed association. ::: #### Expression Translation If the expression provided as annotation value is more complex than just a reference, the OData backend translates CDS expressions to the corresponding OData expression syntax and rejects those expressions that are not applicable in an OData API. ::: info While the flattening of references described in the section above is applied to all annotations, the syntactic translation of expressions is only done for annotations defined in one of the [OData vocabularies](#vocabularies). ::: The following operators and clauses of CDL are supported: * `case when ... then ... else ...` and the logical ternary operator ` ? : ` * Logical: `and`, `or`, `not` * Relational: `=`, `<>`, `!=`, `<`, `<=`, `>`, `>=`, `in`, `between ... and ...` * Unary `+` and `-` * Arithmetic: `+`, `-`, `*`, `/` * Concat: `||` * `cast(...)` Example: ```cds @Some.Xpr: ( -(a + b) ) ``` ```xml a b ``` Such expressions can for example be used for [some Fiori UI annotations](https://ui5.sap.com/#/topic/0e7b890677c240b8ba65f8e8d417c048): ```cds service S { @UI.LineItem: [ // ... { Value: (status), Criticality: ( status = 'O' ? 2 : ( status = 'A' ? 3 : 0 ) ) }] entity Order { key id : Integer; // ... status : String; } } ``` If you need to access an element of an entity in an annotation for a bound action or function, use a path that navigates via an explicitly defined [binding parameter](../cds/cdl#bound-actions). Example: ```cds service S { entity Order { key id : Integer; // ... status : String; } actions { @Core.OperationAvailable: ( :in.status <> 'A' ) action accept (in: $self) } } ``` In addition, the following functions are supported: * `$Null()` representing the `null` value [`Null`]([annotation expression](#null-value)). * `Div(...)` (or `$Div(...)`) and `Mod(...)` (or `$Mod(...)`) for integer division and modulo * [`Has(...)`](https://docs.oasis-open.org/odata/odata/v4.02/csd01/part2-url-conventions/odata-v4.02-csd01-part2-url-conventions.html#Has) (or `$Has(...)`) * the functions listed in sections [5.1.1.5](https://docs.oasis-open.org/odata/odata/v4.02/csd01/part2-url-conventions/odata-v4.02-csd01-part2-url-conventions.html#StringandCollectionFunctions) through [5.1.1.11](https://docs.oasis-open.org/odata/odata/v4.02/csd01/part2-url-conventions/odata-v4.02-csd01-part2-url-conventions.html#GeoFunctions) of [OData URL conventions](https://docs.oasis-open.org/odata/odata/v4.02/odata-v4.02-part2-url-conventions.html) + See examples below for the syntax for `cast` and `isof` (section [5.1.1.10](https://docs.oasis-open.org/odata/odata/v4.02/csd01/part2-url-conventions/odata-v4.02-csd01-part2-url-conventions.html#TypeFunctions)) + The names of the geo functions (section [5.1.1.11](https://docs.oasis-open.org/odata/odata/v4.02/csd01/part2-url-conventions/odata-v4.02-csd01-part2-url-conventions.html#GeoFunctions)) need to be escaped like
`![geo.distance]` * [`fillUriTemplate(...)`](https://docs.oasis-open.org/odata/odata-csdl-xml/v4.01/odata-csdl-xml-v4.01.html#sec_FunctionodatafillUriTemplate) and [`uriEncode(...)`](https://docs.oasis-open.org/odata/odata-csdl-xml/v4.01/odata-csdl-xml-v4.01.html#sec_FunctionodatauriEncode) * `Type(...)` (or `$Type(...)`) is to be used to specify a type name with their corresponding type facets such as `MaxLength(...)`, `Precision(...)`, `Scale(...)` and `SRID(...)` (or `$MaxLength(...)`, `$Precision(...)`, `$Scale(...)`, `$SRID(...)`) Example: ```cds @Some.Func1: ( concat(a, b, c) ) @Some.Func2: ( round(aNumber) ) @Some.Func3: ( $Cast(aValue, $Type('Edm.Decimal', $Precision(38), $Scale(19)) ) ) @Some.Func4: ( $IsOf(aValue, $Type('Edm.Decimal', $Precision(38), $Scale(19)) ) ) @Some.Func5: ( ![geo.distance](a, b) ) @Some.Func6: ( fillUriTemplate(a, b) ) ``` If a functional expression starts with a `$`, all inner function must also be `$` functions and vice versa. Instead of `[$]Type(...)` an EDM primitive type name can be directly used as function name like in CDL. It is worth to mention that there are two alternatives for the cast function, one in the EDM and one in the CDS domain: ```cds @Some.ODataStyleCast: ( Cast(aValue, Decimal(38, 'variable') ) ) // => Edm.Decimal @Some.ODataStyleCast2: ( Cast(aValue, PrimitiveType()) ) // => Edm.PrimitiveType @Some.SQLStyleCast: ( cast(aValue as Decimal(38, variable)) ) // => cds.Decimal @Some.SQLStyleCast2: ( cast(aValue as String) ) // => cds.String without type facets ``` Both `cast` functions look similar, but there are some differences: The OData style `Cast` _function_ starts with a capital letter and the SQL `cast` _operator_ uses the keyword `as` to delimit the element reference from the type specifier. The OData `Cast` requires an EDM primitive type to be used either as `[$]Type()` or as direct type function whereas the SQL `cast` requires a scalar CDS type as argument which is then converted into the corresponding EDM primitive type. ::: info CAP only provides a syntactic translation. It is up to each client whether an expression value is supported for a particular annotation. See for example [Firori's list of supported annotations](https://ui5.sap.com/#/topic/0e7b890677c240b8ba65f8e8d417c048). ::: Use a [dynamic expression](#dynamic-expressions) if the desired EDMX expression cannot be obtained via the automatic translation of a CDS expression. ### Annotating Annotations { #annotating-annotations} OData can annotate annotations. This often occurs in combination with enums like `UI.Importance` and `UI.TextArrangement`. CDS has no corresponding language feature. For OData annotations, nesting can be achieved in the following way: * To annotate a Record, add an additional element to the CDS source structure. The name of this element is the full name of the annotation, including the `@`. See `@UI.Importance` in the following example. * To annotate a single value or a Collection, add a parallel annotation that has the nested annotation name appended to the outer annotation name. See `@UI.Criticality` and `@UI.TextArrangement` in the following example. ```cds @UI.LineItem: [ {Value: ApplicationName, @UI.Importance: #High}, {Value: Description}, {Value: SourceName}, {Value: ChangedBy}, {Value: ChangedAt} ] @UI.LineItem.@UI.Criticality: #Positive @Common.Text: Text @Common.Text.@UI.TextArrangement: #TextOnly ``` Alternatively, annotating a single value or a Collection by turning them into a structure with an artificial property `$value` is still possible, but deprecated: ```cds @UI.LineItem: { $value:[ /* ... */ ], @UI.Criticality: #Positive } @Common.Text: { $value: Text, @UI.TextArrangement: #TextOnly } ``` As `TextArrangement` is common, there's a shortcut for this specific situation: ```cds ... @Common: { Text: Text, TextArrangement: #TextOnly } ``` In any case, the resulting EDMX is: ```xml ... ``` ### Dynamic Expressions { #dynamic-expressions} OData supports dynamic expressions in annotations. For OData annotations you can use the "edm-json inline mechanism" by providing a [dynamic expression](https://docs.oasis-open.org/odata/odata-csdl-json/v4.01/odata-csdl-json-v4.01.html#_Toc38466479) as defined in the [JSON representation of the OData Common Schema Language](https://docs.oasis-open.org/odata/odata-csdl-json/v4.01/odata-csdl-json-v4.01.html) enclosed in `{ $edmJson: { ... }}`. Note that here the CDS syntax for string literals with single quotes (`'foo'`) applies, and that paths are not automatically recognized but need to be written as `{$Path: 'fieldName'}`. The CDS compiler translates the expression into the corresponding [XML representation](https://docs.oasis-open.org/odata/odata-csdl-xml/v4.01/odata-csdl-xml-v4.01.html#_Toc38530421). For example, the CDS annotation: ```cds @UI.Hidden: {$edmJson: {$Ne: [{$Path: 'status'}, 'visible']}} ``` is translated to: ```xml status visible ``` One of the main use cases for such dynamic expressions is SAP Fiori, but note that SAP Fiori supports dynamic expressions only for [specific annotations](https://ui5.sap.com/#/topic/0e7b890677c240b8ba65f8e8d417c048). ::: tip Use expression-like annotation values Instead of writing annotations directly with EDM JSON syntax, try using [expression-like annotation values](#expression-annotations), which are automatically translated. For the example above you would simply write `@UI.Hidden: (status <> 'visible')`. ::: ### `sap:` Annotations In general, back ends and SAP Fiori UIs understand or even expect OData V4 annotations. You should use those rather than the OData V2 SAP extensions.

If necessary, CDS automatically translates OData V4 annotations to OData V2 SAP extensions when invoked with `v2` as the OData version. This means that you shouldn't have to deal with this at all. Nevertheless, in case you need to do so, you can add `sap:...` attribute-style annotations as follows: ```cds @sap.applicable.path: 'to_eventStatus/EditEnabled' action EditEvent(...) returns SomeType; ``` Which would render to OData EDMX as follows: ```xml ... ``` The rules are: * Only strings are supported as values. * The first dot in `@sap.` is replaced by a colon `:`. * Subsequent dots are replaced by dashes. ### Differences to ABAP In contrast to ABAP CDS, we apply a **generic, isomorphic approach** where names and positions of annotations are exactly as specified in the [OData Vocabularies](#vocabularies). This has the following advantages: * Single source of truth — users only need to consult the official OData specs * Speed — we don't need complex case-by-case mapping logic * No bottlenecks — we always support the full set of OData annotations * Bidirectional mapping — we can translate CDS to EDMX and vice versa Last but not least, it also saves us lots of effort as we don't have to write derivatives of all the OData vocabulary specs. ## Annotation Vocabularies { #vocabularies} When translating a CDS model to an OData API, by default only those annotations are considered that are part of the standard OASIS or SAP vocabularies listed below. You can add further vocabularies to the translation process [using configuration.](#additional-vocabularies) ### [OASIS Vocabularies](https://github.com/oasis-tcs/odata-vocabularies#further-description-of-this-repository) { target="_blank"} | Vocabulary | Description | | ------------------------------------------------------------------ | -------------------------------------------- | | [@Aggregation](https://github.com/oasis-tcs/odata-vocabularies/tree/main/vocabularies/Org.OData.Aggregation.V1.md){target="_blank"} | for describing aggregatable data | | [@Authorization](https://github.com/oasis-tcs/odata-vocabularies/tree/main/vocabularies/Org.OData.Authorization.V1.md){target="_blank"} | for authorization requirements | | [@Capabilities](https://github.com/oasis-tcs/odata-vocabularies/tree/main/vocabularies/Org.OData.Capabilities.V1.md){target="_blank"} | for restricting capabilities of a service | | [@Core](https://github.com/oasis-tcs/odata-vocabularies/tree/main/vocabularies/Org.OData.Core.V1.md){target="_blank"} | for general purpose annotations | | [@JSON](https://github.com/oasis-tcs/odata-vocabularies/tree/main/vocabularies/Org.OData.JSON.V1.md){target="_blank"} | for JSON properties | | [@Measures](https://github.com/oasis-tcs/odata-vocabularies/tree/main/vocabularies/Org.OData.Measures.V1.md){target="_blank"} | for monetary amounts and measured quantities | | [@Repeatability](https://github.com/oasis-tcs/odata-vocabularies/tree/main/vocabularies/Org.OData.Repeatability.V1.md){target="_blank"} | for repeatable requests | | [@Temporal](https://github.com/oasis-tcs/odata-vocabularies/tree/main/vocabularies/Org.OData.Temporal.V1.md){target="_blank"} | for temporal annotations | | [@Validation](https://github.com/oasis-tcs/odata-vocabularies/tree/main/vocabularies/Org.OData.Validation.V1.md){target="_blank"} | for adding validation rules | ### [SAP Vocabularies](https://github.com/SAP/odata-vocabularies#readme){target="_blank"} | Vocabulary | Description | | ------------------------------------------------------------- | ------------------------------------------------- | | [@Analytics](https://github.com/SAP/odata-vocabularies/tree/main/vocabularies/Analytics.md){target="_blank"} | for annotating analytical resources | | [@CodeList](https://github.com/SAP/odata-vocabularies/tree/main/vocabularies/CodeList.md){target="_blank"} | for code lists | | [@Common](https://github.com/SAP/odata-vocabularies/tree/main/vocabularies/Common.md){target="_blank"} | for all SAP vocabularies | | [@Communication](https://github.com/SAP/odata-vocabularies/tree/main/vocabularies/Communication.md){target="_blank"} | for annotating communication-relevant information | | [@DataIntegration](https://github.com/SAP/odata-vocabularies/tree/main/vocabularies/DataIntegration.md){target="_blank"} | for data integration | | [@PDF](https://github.com/SAP/odata-vocabularies/tree/main/vocabularies/PDF.md){target="_blank"} | for PDF | | [@PersonalData](https://github.com/SAP/odata-vocabularies/tree/main/vocabularies/PersonalData.md){target="_blank"} | for annotating personal data | | [@Session](https://github.com/SAP/odata-vocabularies/tree/main/vocabularies/Session.md){target="_blank"} | for sticky sessions for data modification | | [@UI](https://github.com/SAP/odata-vocabularies/tree/main/vocabularies/UI.md){target="_blank"} | for presenting data in user interfaces | [Learn more about annotations in CDS and OData and how they work together](https://github.com/SAP-samples/odata-basics-handsonsapdev/blob/annotations/bookshop/README.md){.learn-more} ### Additional Vocabularies Assuming you have a vocabulary `com.MyCompany.vocabularies.MyVocabulary.v1`, you can set the following configuration option: ::: code-group ```json [package.json] { "cds": { "cdsc": { "odataVocabularies": { "MyVocabulary": { "Alias": "MyVocabulary", "Namespace": "com.sap.vocabularies.MyVocabulary.v1", "Uri": "" } } } } } ``` ```json [.cdsrc.json] { "cdsc": { "odataVocabularies": { "MyVocabulary": { "Alias": "MyVocabulary", "Namespace": "com.sap.vocabularies.MyVocabulary.v1", "Uri": "" } } } } ``` ::: With this configuration, all annotations prefixed with `MyVocabulary` are considered in the translation. ```cds service S { @MyVocabulary.MyAnno: 'My new Annotation' entity E { /*...*/ }; }; ``` The annotation is added to the OData API, as well as the mandatory reference to the vocabulary definition: ```xml ... ``` The compiler evaluates neither annotation values nor the URI. It is your responsibility to make the URI accessible if required. Unlike for the standard vocabularies listed above, the compiler has no access to the content of the vocabulary, so the values are translated completely generically. ## Data Aggregation Data aggregation in OData V4 is leveraged by the `$apply` system query option, which defines a pipeline of transformations that is applied to the _input set_ specified by the URI. On the _result set_ of the pipeline, the standard system query options come into effect.

### Example ```http GET /Orders(10)/books? $apply=filter(year eq 2000)/ groupby((author/name),aggregate(price with average as avg))/ orderby(title)/ top(3) ``` This request operates on the books of the order with ID 10. First it filters out the books from the year 2000 to an intermediate result set. The intermediate result set is then grouped by author name and the price is averaged. Finally, the result set is sorted by title and only the top 3 entries are retained. ::: warning If the `groupby` transformation only includes a subset of the entity keys, the result order might be unstable. ::: ### Transformations | Transformation | Description | Node.js | Java | |-------------------------------|----------------------------------------------|---------|--------| | `filter` | filter by filter expression | | | | `search` | filter by search term or expression | | | | `groupby` | group by dimensions and aggregates values | | | | `aggregate` | aggregate values | | | | `compute` | add computed properties to the result set | | | | `expand` | expand navigation properties | | | | `concat` | append additional aggregation to the result | ⁽¹⁾ | | | `skip` / `top` | paginate | ⁽¹⁾ | | | `orderby` | sort the input set | ⁽¹⁾ | | | `topcount`/`bottomcount` | retain highest/lowest _n_ values | | | | `toppercent`/`bottompercent` | retain highest/lowest _p_% values | | | | `topsum`/`bottomsum` | retain _n_ values limited by sum | | | - ⁽¹⁾ Supported with experimental feature `cds.features.odata_new_parser = true` #### `concat` The [`concat` transformation](https://docs.oasis-open.org/odata/odata-data-aggregation-ext/v4.0/cs02/odata-data-aggregation-ext-v4.0-cs02.html#_Toc435016581) applies additional transformation sequences to the input set and concatenates the result: ```http GET /Books?$apply= filter(author/name eq 'Bram Stroker')/ concat( aggregate($count as totalCount), groupby((year), aggregate($count as countPerYear))) ``` This request filters all books, keeping only books by Bram Stroker. From these books, `concat` calculates (1) the total count of books *and* (2) the count of books per year. The result is heterogeneous. The `concat` transformation must be the last of the apply pipeline. If `concat` is used, then `$apply` can't be used in combination with other system query options. #### `skip`, `top`, and `orderby` Beyond the standard transformations specified by OData, CDS Java supports the transformations `skip`, `top`, and `orderby` that allow you to sort and paginate an input set: ```http GET /Order(10)/books? $apply=orderby(price desc)/ top(500)/ groupby((author/name),aggregate(price with max as maxPrice)) ``` This query groups the 500 most expensive books by author name and determines the price of the most expensive book per author. ### Hierarchical Transformations Provide support for hierarchy attribute calculation and navigation, and allow the execution of typical hierarchy operations directly on relational data. | Transformation | Description | Node.js | Java | |-----------------------------------------------|--------------------------------------------------------------------|---------|--------------------| | `com.sap.vocabularies.Hierarchy.v1.TopLevels` | generate a hierarchy based on recursive parent-child source data | | ⁽¹⁾ | | `ancestors` | return all ancestors of a set of start nodes in a hierarchy | | ⁽¹⁾ | | `descendants` | return all descendants of a set of start nodes in a hierarchy | | ⁽¹⁾ | - ⁽¹⁾ Beta feature, API may change ::: warning Generic implementation is supported on SAP HANA only ::: :::info The source elements of the entity defining the recursive parent-child relation are identified by a naming convention or aliases `node_id` and `parent_id`. For more refer to [SAP HANA Hierarchy Developer Guide](https://help.sap.com/docs/SAP_HANA_PLATFORM/4f9859d273254e04af6ab3e9ea3af286/f29c70e984254a6f8df76ad84e78f123.html?locale=en-US&version=2.0.05) ::: #### `com.sap.vocabularies.Hierarchy.v1.TopLevels` The [`TopLevels` transformation](https://github.com/SAP/odata-vocabularies/blob/main/vocabularies/Hierarchy.xml) produces the hierarchical result based on recursive parent-child relationship: ```http GET /SalesOrganizations?$apply= com.sap.vocabularies.Hierarchy.v1.TopLevels(..., NodeProperty='ID', Levels=2) ``` #### `ancestors` and `descendants` The [`ancestors` and `descendants` transformations](https://docs.oasis-open.org/odata/odata-data-aggregation-ext/v4.0/cs03/odata-data-aggregation-ext-v4.0-cs03.html#Transformationsancestorsanddescendants) compute the subset of a given recursive hierarchy, which contains all nodes that are ancestors or descendants of a start nodes set. Its output is the ancestors or descendants set correspondingly. ```http GET SalesOrganizations?$apply= descendants(..., ID, filter(ID eq 'US'), keep start) /ancestors(..., ID, filter(contains(Name, 'New York')), keep start) ``` ### Aggregation Methods | Aggregation Method | Description | Node.js | Java | |-------------------------------|--------------------|---------|--------| | `min` | smallest value | | | | `max` | largest | | | | `sum` | sum of values | | | | `average` | average of values | | | | `countdistinct` | count of distinct values | | | | custom method | custom aggregation method | | | | `$count` | number of instances in input set | | | ### Custom Aggregates Instead of explicitly using an expression with an aggregation method in the `aggregate` transformation, the client can use a _custom aggregate_. A custom aggregate can be considered as a virtual property that aggregates the input set. It's calculated on the server side. The client doesn't know _How_ the custom aggregate is calculated. They can only be used for the special case when a default aggregation method can be specified declaratively on the server side for a measure. A custom aggregate is declared in the CDS model as follows: * The measure must be annotated with an `@Aggregation.default` annotation that specifies the aggregation method. * The CDS entity should be annotated with an `@Aggregation.CustomAggregate` annotation to expose the custom aggregate to the client. ```cds @Aggregation.CustomAggregate#stock : 'Edm.Decimal' entity Books as projection on bookshop.Books { ID, title, @Aggregation.default: #SUM stock }; ``` With this definition, it's now possible to use the custom aggregate `stock` in an `aggregate` transformation: ```http GET /Books?$apply=aggregate(stock) HTTP/1.1 ``` which is equivalent to: ```http GET /Books?$apply=aggregate(stock with sum as stock) HTTP/1.1 ``` #### Currencies and Units of Measure If a property represents a monetary amount, it may have a related property that indicates the amount's *currency code*. Analogously, a property representing a measured quantity can be related to a *unit of measure*. To indicate that a property is a currency code or a unit of measure it can be annotated with the [Semantics Annotations](https://help.sap.com/docs/SAP_NETWEAVER_750/cc0c305d2fab47bd808adcad3ca7ee9d/fbcd3a59a94148f6adad80b9c97304ff.html) `@Semantics.currencyCode` or `@Semantics.unitOfMeasure`. ```cds @Aggregation.CustomAggregate#amount : 'Edm.Decimal' @Aggregation.CustomAggregate#currency : 'Edm.String' entity Sales { key id : GUID; productId : GUID; @Semantics.amount.currencyCode: 'currency' amount : Decimal(10,2); @Semantics.currencyCode currency : String(3); } ``` The CAP Java SDK exposes all properties annotated with `@Semantics.currencyCode` or `@Semantics.unitOfMeasure` as a [custom aggregate](../advanced/odata#custom-aggregates) with the property's name that returns: * The property's value if it's unique within a group of dimensions * `null` otherwise A custom aggregate for a currency code or unit of measure should also be exposed by the `@Aggregation.CustomAggregate` annotation. Moreover, a property for a monetary amount or a measured quantity should be annotated with `@Semantics.amount.currencyCode` or `@Semantics.quantity.unitOfMeasure` to reference the corresponding property that holds the amount's currency code or the quantity's unit of measure, respectively. ### Other Features | Feature | Node.js | Java | |-----------------------------------------|---------|--------| | use path expressions in transformations | | | | chain transformations | | | | chain transformations within group by | | | | `groupby` with `rollup`/`$all` | | | | `$expand` result set of `$apply` | | | | `$filter`/`$search` result set | | | | sort result set with `$orderby` | | | | paginate result set with `$top`/`$skip` | | | ## Open Types An entity type or a complex type may be declared as _open_, allowing clients to add properties dynamically to instances of the type by specifying uniquely named property values in the payload used to insert or update an instance of the type. To indicate that the entity or complex type is open, the corresponding type must be annotated with `@open`: ```cds service CatalogService { @open // [!code focus] entity Book { // [!code focus] key id : Integer; // [!code focus] } // [!code focus] } ``` The cds build for OData v4 will render the entity type `Book` in `edmx` with the [`OpenType` attribute](https://docs.oasis-open.org/odata/odata-csdl-xml/v4.01/odata-csdl-xml-v4.01.html#sec_OpenEntityType) set to `true`: ```xml // [!code focus] ``` The entity `Book` is open, allowing the client to enrich the entity with additional properties. Example 1: ```json {"id": 1, "title": "Tow Sawyer"} ``` Example 2: ```json {"title": "Tow Sawyer", "author": { "name": "Mark Twain", "age": 74 } } ``` Open types can also be referenced in non-open types and entities. This, however, doesn't make the referencing entity or type open. ```cds service CatalogService { type Order { guid: Integer; book: Book; } @open // [!code focus] type Book {} // [!code focus] } ``` The following payload for `Order` is allowed: `{"guid": 1, "book": {"id": 2, "title": "Tow Sawyer"}}` Note that type `Order` itself is not open thus doesn't allow dynamic properties, in contrast to type `Book`. ::: warning Dynamic properties are not persisted in the underlying data source automatically and must be handled completely by custom code. ::: ### Java Type Mapping #### Simple Types The simple values of deserialized JSON payload can be of type: `String`, `Boolean`, `Number` or simply an `Object` for `null` values. |JSON | Java Type of the `value` | |-------------------------|--------------------------------| |`{"value": "Tom Sawyer"}`| `java.lang.String` | |`{"value": true}` | `java.lang.Boolean` | |`{"value": 42}` | `java.lang.Number` (Integer) | |`{"value": 36.6}` | `java.lang.Number` (BigDecimal)| |`{"value": null}` | `java.lang.Object` | #### Structured Types The complex and structured types are deserialized to `java.util.Map`, whereas collections are deserialized to `java.util.List`. |JSON | Java Type of the `value` | |-------------------------------------------------------------------|--------------------------------------| |`{"value": {"name": "Mark Twain"}}` | `java.util.Map` | |`{"value":[{"name": "Mark Twain"}, {"name": "Charlotte Bronte"}}]}`| `java.util.List>`| ## Singletons A singleton is a special one-element entity introduced in OData V4. It can be addressed directly by its name from the service root without specifying the entity's keys. Annotate an entity with `@odata.singleton` or `@odata.singleton.nullable`, to use it as a singleton within a service, for example: ```cds service Sue { @odata.singleton entity MySingleton { key id : String; // can be omitted in OData v4.01 prop : String; assoc : Association to myEntity; } } ``` It can also be defined as an ordered `SELECT` from another entity: ```cds service Sue { @odata.singleton entity OldestEmployee as select from Employees order by birthyear; } ``` ### Requesting Singletons As mentioned above, singletons are accessed without specifying keys in the request URL. They can contain navigation properties, and other entities can include singletons as their navigation properties as well. The `$expand` query option is also supported. ```http GET …/MySingleton GET …/MySingleton/prop GET …/MySingleton/assoc GET …/MySingleton?$expand=assoc ``` ### Updating Singletons The following request updates a prop property of a singleton MySingleton: ```http PATCH/PUT …/MySingleton {prop: “New value”} ``` ### Deleting Singletons A `DELETE` request to a singleton is possible only if a singleton is annotated with `@odata.singleton.nullable`. An attempt to delete a singleton annotated with `@odata.singleton` will result in an error. ### Creating Singletons Since singletons represent a one-element entity, a `POST` request is not supported.

## V2 Support While CAP defaults to OData V4, the latest protocol version, older projects may need to fallback to OData V2, for example, to keep using existing V2-based UIs. ::: warning OData V2 is deprecated. Use OData V2 only if you need to support existing UIs or if you need to use specific controls that don't work with V4 **yet** like, tree tables (sap.ui.table.TreeTable). ::: ### Enabling OData V2 via CDS OData V2 Adapter in Node.js Apps { #odata-v2-adapter-node} CAP Node.js supports serving the OData V2 protocol through the [_OData V2 adapter for CDS_](https://www.npmjs.com/package/@cap-js-community/odata-v2-adapter), which translates between the OData V2 and V4 protocols. For Node.js projects, add the CDS OData V2 adapter as express.js middleware as follows: 1. Add the adapter package to your project: ```sh npm add @cap-js-community/odata-v2-adapter ``` 2. Access OData V2 services at [http://localhost:4004/odata/v2/${path}](http://localhost:4004/odata/v2). 3. Access OData V4 services at [http://localhost:4004/odata/v4/${path}](http://localhost:4004/odata/v4) (as before). Example: Read service metadata for `CatalogService`: - CDS: ```cds @path:'/browse' service CatalogService { ... } ``` - OData V2: `GET http://localhost:4004/odata/v2/browse/$metadata` - OData V4: `GET http://localhost:4004/odata/v4/browse/$metadata` [Find detailed instructions at **@cap-js-community/odata-v2-adapter**.](https://www.npmjs.com/package/@cap-js-community/odata-v2-adapter){.learn-more} ### Using OData V2 in Java Apps In CAP Java, serving the OData V2 protocol is supported natively by the [CDS OData V2 Adapter](../java/migration#v2adapter). ## Miscellaneous ### Omitting Elements from APIs Add annotation `@cds.api.ignore` to suppress unwanted entity fields (for example, foreign-key fields) in APIs exposed from this the CDS model, that is, OData or OpenAPI. For example: ```cds entity Books { ... @cds.api.ignore author : Association to Authors; } ``` Please note that `@cds.api.ignore` is effective on regular elements that are rendered as `Edm.Property` only. The annotation doesn't suppress an `Edm.NavigationProperty` which is rendered for associations or compositions. If a managed association is annotated, the annotations are propagated to the (generated) foreign keys. In the previous example, the foreign keys of the managed association `author` are muted in the API. ### Absolute Context URL { #absolute-context-url} In some scenarios, an absolute [context URL](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_ContextURL) is needed. In the Node.js runtime, this can be achieved through configuration `cds.odata.contextAbsoluteUrl`. You can use your own URL (including a protocol and a service path), for example: ```js cds.odata.contextAbsoluteUrl = "https://your.domain.com/yourService" ``` to customize the annotation as follows: ```json { "@odata.context":"https://your.domain.com/yourService/$metadata#Books(title,author,ID)", "value":[ {"ID": 201,"title": "Wuthering Heights","author": "Emily Brontë"}, {"ID": 207,"title": "Jane Eyre","author": "Charlotte Brontë"}, {"ID": 251,"title": "The Raven","author": "Edgar Allen Poe"} ] } ``` If `contextAbsoluteUrl` is set to something truthy that doesn't match `http(s)://*`, an absolute path is constructed based on the environment of the application on a best effort basis. Note that we encourage you to stay with the default relative format, if possible, as it's proxy safe. # Publishing to AsyncAPI You can convert events in CDS models to the [AsyncAPI specification](https://www.asyncapi.com), a widely adopted standard used to describe and document message-driven asynchronous APIs. ## Usage from CLI { #cli} Use the following command to convert all services in `srv/` and store the generated AsyncAPI documents in the `docs/` folder: ```sh cds compile srv --service all -o docs --to asyncapi ``` For each service that is available in the `srv/` files, an AsyncAPI document with the service name is generated in the output folder. If you want to generate one AsyncAPI document for all the services, you can use `--asyncapi:merged` flag: ```sh cds compile srv --service all -o docs --to asyncapi --asyncapi:merged ``` [Learn how to programmatically convert the CSN file into an AsyncAPI Document](/node.js/cds-compile#to-asyncapi){.learn-more} ## Presets { #presets} Use presets to add configuration for the AsyncAPI export tooling. ::: code-group ```json [.cdsrc.json] { "export": { "asyncapi": { "application_namespace": "sap.example" [...] } } } ``` ::: | Term | Preset Target | AsyncAPI field | Remarks | |-------------------------|---------------|-------------------------------|--------------------------------------------------------------------------------------------------------------------------| | `merged.title` | Service | info.title | Mandatory when `--asyncapi:merged` flag is given.
`title` from here is used in the generated AsyncAPI document. | | `merged.version` | Service | info.version | Mandatory when `--asyncapi:merged` flag is given.
`version` from here is used in the generated AsyncAPI document | | `merged.description` | Service | info.description | Optional when `--asyncapi:merged` flag is given.
`description` from here is used in the generated AsyncAPI document. | | `merged.short_text` | Service | x-sap-shortText | Optional when `--asyncapi:merged` flag is given.
The value from here is used in the generated AsyncAPI document. | | `application_namespace` | Document | x-sap-application-namespace | Mandatory | | `event_spec_version` | Event | x-sap-event-spec-version | | | `event_source` | Event | x-sap-event-source | | | `event_source_params` | Event | x-sap-event-source-parameters | | | `event_characteristics` | Event | x-sap-event-characteristics | | ## Annotations { #annotations} Use annotations to add configuration for the AsyncAPI export tooling. ::: tip Annotations will take precedence over [presets](#presets). ::: | Term (`@AsyncAPI.`) | Annotation Target | AsyncAPI field | Remarks | |------------------------|-------------------|-------------------------------|-------------------------------------------------------------------------------------------------------------------------| | `Title` | Service | info.title | Mandatory | | `SchemaVersion` | Service | info.version | Mandatory | | `Description` | Service | info.description | | | `StateInfo` | Service | x-sap-stateInfo | | | `ShortText` | Service | x-sap-shortText | | | `EventSpecVersion` | Event | x-sap-event-spec-version | | | `EventSource` | Event | x-sap-event-source | | | `EventSourceParams` | Event | x-sap-event-source-parameters | | | `EventCharacteristics` | Event | x-sap-event-characteristics | | | `EventStateInfo` | Event | x-sap-stateInfo | | | `EventSchemaVersion` | Event | x-sap-event-version | | | `EventType` | Event | | Optional; The value from this annotation will be used to
overwrite the default event type in the AsyncAPI document. | For example: ```cds @AsyncAPI.Title : 'CatalogService Events' @AsyncAPI.SchemaVersion: '1.0.0' @AsyncAPI.Description : 'Events emitted by the CatalogService.' service CatalogService { @AsyncAPI.EventSpecVersion : '2.0' @AsyncAPI.EventCharacteristics: { ![state-transfer]: 'full-after-image' } @AsyncAPI.EventSchemaVersion : '1.0.0' event SampleEntity.Changed.v1 : projection on CatalogService.SampleEntity; } ``` ## Type Mapping { #mapping} CDS Type to AsyncAPI Mapping | CDS Type | AsyncAPI Supported Types | |----------------------------------------|-----------------------------------------------------------------------------------------------------| | `UUID` | `{ "type": "string", "format": "uuid" }` | | `Boolean` | `{ "type": "boolean" }` | | `Integer` | `{ "type": "integer" }` | | `Integer64` | `{ "type": "string", "format": "int64" }` | | `Decimal`, `{precision, scale}` | `{ "type": "string", "format": "decimal", "formatPrecision": , "formatScale": }` | | `Decimal`, without scale | `{ "type": "string", "format": "decimal", "formatPrecision": }` | | `Decimal`, without precision and scale | `{ "type": "string", "format": "decimal" }` | | `Double` | `{ "type": "number" }` | | `Date` | `{ "type": "string", "format": "date" }` | | `Time` | `{ "type": "string", "format": "partial-time" }` | | `DateTime` | `{ "type": "string", "format": "date-time" }` | | `Timestamp` | `{ "type": "string", "format": "date-time" }` | | `String` | `{ "type": "string", "maxLength": length }` | | `Binary` | `{ "type": "string", "maxLength": length }` | | `LargeBinary` | `{ "type": "string" }` | | `LargeString` | `{ "type": "string" }` | # Serving Fiori UIs CAP provides out-of-the-box support for SAP Fiori elements front ends. This guide explains how to add one or more SAP Fiori elements apps to a CAP project, how to add SAP Fiori elements annotations to respective service definitions, and more. In the following sections, when mentioning Fiori, we always mean SAP Fiori elements. [Learn more about developing SAP Fiori elements and OData V4 (since 1.84.)](https://sapui5.hana.ondemand.com/#/topic/62d3f7c2a9424864921184fd6c7002eb){.learn-more} ## SAP Fiori Preview For entities exposed via OData V4 there is a _Fiori preview_ link on the index page. It dynamically serves an SAP Fiori Elements list page that allows you to quickly see the effect of annotation changes without having to create a UI application first.

::: details Be aware that this is **not meant for production**. The preview not meant as a replacement for a proper SAP Fiori Elements (UI5) application. It is only active locally where the [development profile](../node.js/cds-env#profiles) is enabled. To also enable it in cloud deployments, for test or demo purposes maybe, add the following configuration: ::: code-group ```json [package.json] { "cds": { "features": { "fiori_preview": true } } } ``` ```json [.cdsrc.json] { "features": { "fiori_preview": true } } ``` :::

::: details Be aware that this is **not meant for production**. The preview not meant as a replacement for a proper SAP Fiori Elements (UI5) application. It is active by default, but disabled automatically in case the [production profile](../java/developing-applications/configuring#production-profile) is enabled. To also enable it in cloud deployments, for test or demo purposes maybe, add the following configuration: ::: code-group ```yaml [srv/src/main/resources/application.yaml] cds: index-page: enabled: true ``` :::

## Adding Fiori Apps As showcased in [cap/samples](https://github.com/sap-samples/cloud-cap-samples/tree/main/fiori/app), SAP Fiori apps should be added as sub folders to the `app/` of a CAP project. Each sub folder constitutes an individual SAP Fiori application, with [local annotations](#fiori-annotations), _manifest.json_, etc. So, a typical folder layout would look like this: | Folder/Sub Folder | Description | |----------------------------|--------------------------------------| | `app/` | All SAP Fiori apps should go in here | | `browse/` | SAP Fiori app for end users | | `orders/` | SAP Fiori app for order management | | `admin/` | SAP Fiori app for admins | | `index.html` | For sandbox tests | | `srv/` | All services | | `db/` | Domain models, and db stuff | ::: tip Links to Fiori applications created in the `app/` folder are automatically added to the index page of your CAP application for local development. ::: ### Using SAP Fiori Tools The SAP Fiori tools provide advanced support for adding SAP Fiori apps to existing CAP projects as well as a wealth of productivity tools, for example for adding SAP Fiori annotations, or graphical modeling and editing. They can be used locally in [Visual Studio Code (VS Code)](https://marketplace.visualstudio.com/items?itemName=SAPSE.sap-ux-fiori-tools-extension-pack) or in [SAP Business Application Studio](https://help.sap.com/docs/SAP_FIORI_tools/17d50220bcd848aa854c9c182d65b699/b0110400b44748d7b844bb5977a657fa.html). [Learn more about **how to install SAP Fiori tools**.](https://help.sap.com/docs/SAP_FIORI_tools/17d50220bcd848aa854c9c182d65b699/2d8b1cb11f6541e5ab16f05461c64201.html){.learn-more} ### From [cap/samples](https://github.com/sap-samples/cloud-cap-samples) For example, you can copy the [SAP Fiori apps from cap/samples](https://github.com/sap-samples/cloud-cap-samples/tree/main/fiori/app) as a template and modify the content as appropriate. ### From [Incidents Sample](https://github.com/SAP-samples/fiori-elements-incident-management/tree/sampleSolution) This is a sample to create an incident management app with SAP Fiori elements for OData V4. ## Fiori Annotations The main content to add is service definitions annotated with information about how to render respective data. ### What Are SAP Fiori Annotations? SAP Fiori elements apps are generic front ends, which construct and render the pages and controls based on annotated metadata documents. The annotations provide semantic annotations used to render such content, for example: ```cds annotate CatalogService.Books with @( UI: { SelectionFields: [ ID, price, currency_code ], LineItem: [ {Value: title}, {Value: author, Label:'{i18n>Author}'}, {Value: genre.name}, {Value: price}, {Value: currency.symbol, Label:' '}, ] } ); ``` [Find this source and many more in **cap/samples**.](https://github.com/sap-samples/cloud-cap-samples/tree/main/fiori/app){.learn-more target="_blank"} [Learn more about **OData Annotations in CDS**.](./odata#annotations){.learn-more} ### Where to Put Them? While CDS in principle allows you to add such annotations everywhere in your models, we recommend putting them in separate _.cds_ files placed in your _./app/*_ folders, for example, as follows. ```sh ./app #> all your Fiori annotations should go here, for example: ./admin fiori-service.cds #> annotating ../srv/admin-service.cds ./browse fiori-service.cds #> annotating ../srv/cat-service.cds index.cds ./srv #> all service definitions should stay clean in here: admin-service.cds cat-service.cds ... ``` [See this also in **cap/samples/fiori**.](https://github.com/sap-samples/cloud-cap-samples/tree/main/fiori/app){.learn-more} **Reasoning:** This recommendation essentially follows the best practices and guiding principles of [Conceptual Modeling](../guides/domain-modeling#domain-driven-design) and [Separation of Concerns](../guides/domain-modeling#separation-of-concerns). ### Maintaining Annotations Maintaining OData annotations in _.cds_ files is accelerated by the SAP Fiori tools - CDS OData Language Server [@sap/ux-cds-odata-language-server-extension](https://www.npmjs.com/package/@sap/ux-cds-odata-language-server-extension) in the [SAP CDS language support plugin](https://marketplace.visualstudio.com/items?itemName=SAPSE.vscode-cds). It helps you add and edit OData annotations in CDS syntax with: - Code completion - Validation against the OData vocabularies and project metadata - Navigation to the referenced annotations - Quick view of vocabulary information - Internationalization support These assisting features are provided for [OData annotations in CDS syntax](../advanced/odata#annotations) and can't be used yet for the [core data services common annotations](../cds/annotations). The [@sap/ux-cds-odata-language-server-extension](https://www.npmjs.com/package/@sap/ux-cds-odata-language-server-extension) module doesn't require any manual installation. The latest version is fetched by default from [npmjs.com](https://npmjs.com) as indicated in the user preference setting **CDS > Contributions: Registry**. [Learn more about the **CDS extension for VS Code**.](https://www.youtube.com/watch?v=eY7BTzch8w0){.learn-more} ### Code Completion The CDS OData Language Server provides a list of context-sensitive suggestions based on the service metadata and OData vocabularies. You can use it to choose OData annotation terms, their properties, and values from the list of suggestions in annotate directives applied to service entities and entity elements. See [annotate directives](../cds/cdl#annotate) for more details. #### Using Code Completion To trigger code completion, choose ⌘ + ⎵ (macOS) or Ctrl + ⎵ (other platforms). The list of suggested values is displayed. > Note: You can filter the list of suggested values by typing more characters. Navigate to the desired value using the up or down arrows or your mouse. Accept the highlighted value by pressing Enter or by clicking the mouse. Use code completion to add and change individual values (word-based completion) and to add small code blocks containing annotation structures along with mandatory properties (micro-snippets). In an active code snippet, you can use the ⇥ (tab) key to quickly move to the next tab stop. ##### Example: Annotating Service Entities $cursor position indicated by `|`$ 1. Place cursor in the `annotate` directive for a service entity, for example `annotate Foo.Bar with ;` and trigger code completion. 2. Type `u` to filter the suggestions and choose `{} UI`. Micro-snippet `@UI : {|}` is inserted: `annotate Foo.Bar with @UI : {|};` 3. Use code completion again to add an annotation term from the UI vocabulary, in this example `SelectionFields`. The micro snippet for this annotation is added and the cursor is placed directly after the term name letting you define a qualifier: `annotate Foo.Bar with @UI : {SelectionFields | : []};` 4. Press the ⇥ (tab) key to move the cursor to the next tab stop and use code completion again to add values. Because the `UI.SelectionFields` annotation is a collection of entity elements $entity properties$, all elements of the annotated entity are suggested. ::: tip To choose an element of an associated entity, first select the corresponding association from the list and type *. $period$*. Elements of associated entity are suggested. Note: You can add multiple values separated by comma. ::: ```cds annotate Foo.Bar with @UI : { SelectionFields : [ description, assignedIndividual.lastName| ], }; ``` 5. Add a new line after `,` (comma) and use code completion again to add another annotation from the UI vocabulary, such as `LineItem`. Line item is a collection of `DataField` records. To add a record, select the record type you need from the completion list. ```cds annotate Foo.Bar with @UI : { SelectionFields : [ description, assignedIndividual.lastName ], LineItem : [{ $Type:'UI.DataField', Value : |, }, }; ``` > Note: For each record type, two kinds of micro-snippets are provided: one containing only mandatory properties and one containing all properties defined for this record $full record$. Usually you need just a subset of properties. So, you either select a full record and then remove the properties you don't need, or add the record containing only required properties and then add the remaining properties. 6. Use code completion to add values for the annotation properties. ```cds annotate Foo.Bar with @UI : { SelectionFields : [ description, assignedIndividual.lastName ], LineItem : [ { $Type:'UI.DataField', Value : description, }, { $Type:'UI.DataFieldForAnnotation', Target : 'assignedIndividual/@Communication.Contact', },| ] }; ``` > Note: To add values pointing to annotations defined in another CDS source, you must reference this source with the `using` directive. See [The `using` Directive](../cds/cdl#using) for more details. ##### Example: Annotating Entity Elements $cursor position indicated by `|`$ 1. Place the cursor in the `annotate` directive, for example `annotate Foo.Bar with {|};`, add a new line and trigger code completion. You get the list of entity elements. Choose the one that you want to annotate. ```cds annotate Foo.Bar with { code| }; ``` 2. Press the ⎵ key, use code completion again, and choose `{} UI`. The `@UI : {|}` micro-snippet is inserted: ```cds annotate Foo.Bar with { code @UI : { | } }; ``` 3. Trigger completion again and choose an annotation term from the UI vocabulary, in this example: **Hidden**. ```cds annotate Foo.Bar with { code @UI : {Hidden : |} }; ``` 4. Press the ⇥ (tab) key to move the cursor to the next tab stop and use code completion again to add the value. Because the `UI.Hidden` annotation is of Boolean type, the values true and false are suggested: ```cds annotate Foo.Bar with { code @UI : {Hidden : false } }; ``` ### Diagnostics The CDS OData Language Server validates OData annotations in _.cds_ files against the service metadata and OData vocabularies. It also checks provided string content for language-dependent annotation values and warns you if the format doesn't match the internationalization (i18n) key reference. It shows you that this string is hard coded and won't change based on the language setting in your application. See [Internationalization support](#internationalization-support) for more details. Validation is performed when you open a _.cds_ file and then is retriggered with each change to the relevant files. You can view the diagnostic messages by hovering over the highlighted part in the annotation file or by opening the problems panel. Click on the message in the problems panel to navigate to the related place in the annotation file. > Note: If an annotation value points to the annotation defined in another CDS source, you must reference this source with a `using` directive to avoid warnings. See [The `using` Directive](../cds/cdl#using) for more details. ### Navigation to Referenced Annotations CDS OData Language Server enables quick navigation to the definition of referenced annotations. For example, if your annotation file contains a `DataFieldForAnnotation` record referencing an `Identification` annotation defined in the service file, you can view which file it's defined in and what fields or labels this annotation contains. You can even update the `Identification` annotation or add comments. You can navigate to the referenced annotation using the [Peek Definition](#peek-definition) and [Go To Definition](#go-to-definition) features. > Note: If the referenced annotation is defined in another CDS source, you must reference this source with the `using` directive to enable the navigation. See [The `using` Directive](../cds/cdl#using) for more details. #### Peek Definition { #peek-definition} Peek Definition lets you preview and update the referenced annotation without switching away from the code that you're writing. It's triggered when your cursor is inside the referenced annotation value. - Using a keyboard: choose ⌥ + F12 (macOS) or Alt + F12 (other platforms) - Using a mouse: right-click and select **Peek Definition** If an annotation is defined in multiple sources, all these sources are listed. You can select which one you want to view or update. Annotation layering isn't considered. #### Go to Definition { #go-to-definition} Go To Definition lets you navigate to the source of the referenced annotation and opens the source file scrolled to the respective place in a new tab. It's triggered when your cursor is inside the referenced annotation value. Place your cursor inside the path referencing the annotation term segment or translatable string value, and trigger Go to Definition: - Using a keyboard: choose F12 in VS Code, or Ctrl + F12 in SAP Business Application Studio - Using a mouse: right-click and select **Go To Definition** - Using a keyboard and mouse: ⌘ + mouse click (macOS) or Ctrl + mouse click (other platforms) If an annotation is defined in multiple sources, a Peek definition listing these sources will be shown instead. Annotation layering isn't considered. ### Documentation $Quick Info$ The annotation language server provides quick information for annotation terms, record types, and properties used in the annotation file, or provided as suggestions in code completion lists. This information is retrieved from the respective OData vocabularies and can provide answers to the following questions: - What is the type and purpose of the annotation term/record type/property? - What targets can the annotation term apply to? - Is the annotation term/record type/property experimental? Is it deprecated? - Is this annotation property mandatory or optional? > Note: The exact content depends on the availability in OData vocabularies. To view the quick info for an annotation term, record type, or property used in the annotation file, hover your mouse over it. The accompanying documentation is displayed in a hover window, if provided in the respective OData vocabularies. To view the quick info for each suggestion in the code completion list, either pressing ⌘ + ⎵ (macOS) or Ctrl + ⎵ (other platforms), or click the *info* icon. The accompanying documentation for the suggestion expands to the side. The expanded documentation stays open and updates as you navigate the list. You can close this by pressing ⌘ + ⎵ / Ctrl + ⎵ again or by clicking on the close icon. #### Internationalization Support When you open an annotation file, all language-dependent string values are checked against the _i18n.properties_ file. Each value that doesn't represent a valid reference to the existing text key in the _i18n.properties_ file, is indicated with a warning. A Quick Fix action is suggested to generate a text key in i18n file and substitute your string value with the reference to that entry. ### Prefer `@title` and `@description` Influenced by the [JSON Schema](https://json-schema.org), CDS supports the [common annotations](../cds/annotations#common-annotations) `@title` and `@description`, which are mapped to corresponding [OData annotations](./odata#annotations) as follows: | CDS | JSON Schema | OData | |----------------|---------------|---------------------| | `@title` | `title` | `@Common.Label` | | `@description` | `description` | `@Core.Description` | We recommend preferring these annotations over the OData ones in protocol-agnostic data models and service models, for example: ```cds annotate my.Books with { //... title @title: 'Book Title'; author @title: 'Author ID'; } ``` ### Prefer `@readonly`, `@mandatory`, ... CDS supports `@readonly` as a common annotation, which translates to respective [OData annotations](./odata#annotations) from the `@Capabilities` vocabulary. We recommend using the former for reasons of conciseness and comprehensibility as shown in this example: ```cds @readonly entity Foo { // entity-level @readonly foo : String // element-level } ``` is equivalent to: ```cds entity Foo @(Capabilities:{ // entity-level InsertRestrictions.Insertable: false, UpdateRestrictions.Updatable: false, DeleteRestrictions.Deletable: false }) { // element-level @Core.Computed foo : String } ``` Similar recommendations apply to `@mandatory` and others → see [Common Annotations](../cds/annotations#common-annotations). ## Draft Support SAP Fiori supports edit sessions with draft states stored on the server, so users can interrupt and continue later on, possibly from different places and devices. CAP, as well as SAP Fiori elements, provide out-of-the-box support for drafts as outlined in the following sections. **We recommend to always use draft** when your application needs data input by end users. [For details and guidelines, see **SAP Fiori Design Guidelines for Draft**.](https://experience.sap.com/fiori-design-web/draft-handling/){.learn-more} [Find a working end-to-end version in **cap/samples/fiori**.](https://github.com/sap-samples/cloud-cap-samples/tree/main/fiori){.learn-more} [For details about the draft flow in SAP Fiori elements, see **SAP Fiori elements > Draft Handling**](https://ui5.sap.com/#/topic/ed9aa41c563a44b18701529c8327db4d){.learn-more} ### Enabling Draft with `@odata.draft.enabled` To enable draft for an entity exposed by a service, simply annotate it with `@odata.draft.enabled` as in this example: ```cds annotate AdminService.Books with @odata.draft.enabled; ``` [See it live in **cap/samples**.](https://github.com/sap-samples/cloud-cap-samples/tree/main/fiori/app/admin-books/fiori-service.cds#L51){.learn-more} ::: warning You can't project from draft-enabled entities, as annotations are propagated. Either _enable_ the draft for the projection and not the original entity or _disable_ the draft on the projection using `@odata.draft.enabled: null`. ::: ### Difference between Compositions and Associations Be aware that you must not modify associated entities through drafts. Only compositions will get a "Create" button in SAP Fiori elements UIs because they are stored as part of the same draft entity. ### Enabling Draft for [Localized Data](../guides/localized-data) {#draft-for-localized-data} Annotate the underlying base entity in the base model with `@fiori.draft.enabled` to also support drafts for [localized data](../guides/localized-data): ```cds annotate sap.capire.bookshop.Books with @fiori.draft.enabled; ``` :::info Background SAP Fiori drafts required single keys of type `UUID`, which isn't the case by default for the automatically generated `_texts` entities (→ [see the _Localized Data_ guide for details](../guides/localized-data#behind-the-scenes)). The `@fiori.draft.enabled` annotation tells the compiler to add such a technical primary key element named `ID_texts`. ::: ::: warning Adding the annotation `@fiori.draft.enabled` won't work if the corresponding `_texts` entities contain any entries, because existing entries don't have a value for the new key field `ID_texts`. ::: ![An SAP Fiori UI showing how a book is edited in the bookshop sample and that the translations tab is used for non-standard languages.](../assets/draft-for-localized-data.png){} [See it live in **cap/samples**.](https://github.com/sap-samples/cloud-cap-samples/tree/main/fiori/app/admin-books/fiori-service.cds#L50){.learn-more} If you're editing data in multiple languages, the _General_ tab in the example above is reserved for the default language (often "en"). Any change to other languages has to be done in the _Translations_ tab, where a corresponding language can be chosen from a drop-down menu as illustrated above. This also applies if you use the URL parameter `sap-language` on the draft page. ### Validating Drafts You can add [custom handlers](../guides/providing-services#custom-logic) to add specific validations, as usual. In addition, for a draft, you can register handlers to the `PATCH` events to validate input per field, during the edit session, as follows. ##### ... in Java You can add your validation logic before operation event handlers. Specific events for draft operations exist. See [Java > Fiori Drafts > Editing Drafts](../java/fiori-drafts#draftevents) for more details. ##### ... in Node.js You can add your validation logic before the operation handler for either CRUD or draft-specific events. See [Node.js > Fiori Support > Handlers Registration](../node.js/fiori#draft-support) for more details about handler registration.

### Query Drafts Programmatically To access drafts in code, you can use the [`.drafts` reflection](../node.js/cds-reflect#drafts). ```js SELECT.from(Books.drafts) //returns all drafts of the Books entity ``` [Learn how to query drafts in Java.](../java/fiori-drafts#draftservices){.learn-more} ## Use Roles to Toggle Visibility of UI elements In addition to adding [restrictions on services, entities, and actions/functions](/guides/security/authorization#restrictions), there are use cases where you only want to hide certain parts of the UI for specific users. This is possible by using the respective UI annotations like `@UI.Hidden` or `@UI.CreateHidden` in conjunction with `$edmJson` pointing to a singleton. First, you define the [singleton](../advanced/odata#singletons) in your service and annotate it with [`@cds.persistency.skip`](../guides/databases#cds-persistence-skip) so that no database artefact is created: ```cds @odata.singleton @cds.persistency.skip entity Configuration { key ID: String; isAdmin : Boolean; } ``` > A key is technically not required, but without it some consumers might run into problems. Then define an `on` handler for serving the request: ```js srv.on('READ', 'Configuration', async req => { req.reply({ isAdmin: req.user.is('admin') //admin is the role, which for example is also used in @requires annotation }); }); ``` Finally, refer to the singleton in the annotation by using a [dynamic expression](../advanced/odata#dynamic-expressions): ```cds annotate service.Books with @( UI.CreateHidden : { $edmJson: {$Not: { $Path: '/CatalogService.EntityContainer/Configuration/isAdmin'} } }, UI.UpdateHidden : { $edmJson: {$Not: { $Path: '/CatalogService.EntityContainer/Configuration/isAdmin'} } }, ); ``` The Entity Container is OData specific and refers to the `$metadata` of the OData service in which all accessible entities are located within the Entity Container. :::details SAP Fiori elements also allows to not include it in the path ```cds annotate service.Books with @( UI.CreateHidden : { $edmJson: {$Not: { $Path: '/Configuration/isAdmin'} } }, UI.UpdateHidden : { $edmJson: {$Not: { $Path: '/Configuration/isAdmin'} } }, ); ``` ::: ## Value Helps In addition to supporting the standard `@Common.ValueList` annotations as defined in the [OData Vocabularies](odata#annotations), CAP provides advanced, convenient support for Value Help as understood and supported by SAP Fiori. ### Convenience Option `@cds.odata.valuelist` Simply add the `@cds.odata.valuelist` annotation to an entity, and all managed associations targeting this entity will automatically receive Value Lists in SAP Fiori clients. For example: ```cds @cds.odata.valuelist entity Currencies {} ``` ```cds service BookshopService { entity Books { //... currency : Association to Currencies; } } ``` ### Pre-Defined Types in `@sap/cds/common` [@sap/cds/common]: ../cds/common The reuse types in [@sap/cds/common] already have this added to base types and entities, so all uses automatically benefit from this. This is an effective excerpt of respective definitions in `@sap/cds/common`: ```cds type Currencies : Association to sap.common.Currencies; ``` ```cds context sap.common { entity Currencies : CodeList {...}; entity CodeList { name : localized String; ... } } ``` ```cds annotate sap.common.CodeList with @( UI.Identification: [name], cds.odata.valuelist, ); ``` ### Usages of `@sap/cds/common` In effect, usages of [@sap/cds/common] stay clean of any pollution, for example: ```cds using { Currency } from '@sap/cds/common'; entity Books { //... currency : Currency; } ``` [Find this also in our **cap/samples**.](https://github.com/sap-samples/cloud-cap-samples/tree/main/bookshop/db/schema.cds){.learn-more} Still, all SAP Fiori UIs, on all services exposing `Books`, will automatically receive Value Help for currencies. You can also benefit from that when [deriving your project-specific code list entities from **sap.common.CodeList**](../cds/common#adding-own-code-lists). ### Resulting Annotations in EDMX Here is an example showing how this ends up as OData `Common.ValueList` annotations: ```xml ``` ## Actions In our SFLIGHT sample application, we showcase how to use actions covering the definition in your CDS model, the needed custom code and the UI implementation. [Learn more about Custom Actions & Functions.](../guides/providing-services#actions-functions){.learn-more} We're going to look at three things. 1. Define the action in CDS and custom code. 1. Create buttons to bring the action to the UI 1. Dynamically define the buttons status on the UI First you need to define an action, like in the [_travel-service.cds_ file](https://github.com/SAP-samples/cap-sflight/blob/dfc7827da843ace0ea126f76fc78a6591b325c67/srv/travel-service.cds#L11). ```cds entity Travel as projection on my.Travel actions { action createTravelByTemplate() returns Travel; action rejectTravel(); action acceptTravel(); action deductDiscount( percent: Percentage not null ) returns Travel; }; ``` To define what the action actually is doing, you need to write some custom code. See the [_travel-service.js_](https://github.com/SAP-samples/cap-sflight/blob/dfc7827da843ace0ea126f76fc78a6591b325c67/srv/travel-service.js#L126) file for example: ```js this.on('acceptTravel', req => UPDATE(req._target).with({TravelStatus_code:'A'})) ``` > Note: `req._target` is a workaround that has been [introduced in SFlight](https://github.com/SAP-samples/cap-sflight/blob/685867de9e6a91d61276671e4af7354029c70ac8/srv/workarounds.js#L52). In the future, there might be an official API for it. Create the buttons, to bring this action onto the UI and make it actionable for the user. There are two buttons: On the overview and in the detail screen. Both are defined in the [_layouts.cds_](https://github.com/SAP-samples/cap-sflight/blob/dfc7827da843ace0ea126f76fc78a6591b325c67/app/travel_processor/layouts.cds) file. For the overview of all travels, use the [`@UI.LineItem` annotation](https://github.com/SAP-samples/cap-sflight/blob/dfc7827da843ace0ea126f76fc78a6591b325c67/app/travel_processor/layouts.cds#L40-L41). ```cds annotate TravelService.Travel with @UI : { LineItem : [ { $Type : 'UI.DataFieldForAction', Action : 'TravelService.acceptTravel', Label : '{i18n>AcceptTravel}' } ] }; ``` For the detail screen of a travel, use the [`@UI.Identification` annotation](https://github.com/SAP-samples/cap-sflight/blob/dfc7827da843ace0ea126f76fc78a6591b325c67/app/travel_processor/layouts.cds#L9-L10). ```cds annotate TravelService.Travel with @UI : { Identification : [ { $Type : 'UI.DataFieldForAction', Action : 'TravelService.acceptTravel', Label : '{i18n>AcceptTravel}' } ] }; ``` Now, the buttons are there and connected to the action. The missing piece is to define the availability of the buttons dynamically. Annotate the `Travel` entity in the `TravelService` service accordingly in the [_field-control.cds_](https://github.com/SAP-samples/cap-sflight/blob/8f65dc8b7985bc22584d2a9f94335f110c0450ea/app/travel_processor/field-control.cds#L20-L32) file. ```cds annotate TravelService.Travel with actions { acceptTravel @( Core.OperationAvailable : { $edmJson: { $Ne: [{ $Path: 'in/TravelStatus_code'}, 'A']} }, Common.SideEffects.TargetProperties : ['in/TravelStatus_code'], ) }; ``` This annotation uses [dynamic expressions](../advanced/odata#dynamic-expressions) to control the buttons for each action. And the status of a travel on the UI is updated, triggered by the `@Common.SideEffects.TargetProperties` annotation. :::info More complex calculation If you have the need for a more complex calculation, then the interesting parts in SFLIGHT are [virtual fields in _field-control.cds_](https://github.com/SAP-samples/cap-sflight/blob/dfc7827da843ace0ea126f76fc78a6591b325c67/app/travel_processor/field-control.cds#L10-L16) (also lines 37-44) and [custom code in _travel-service.js_](https://github.com/SAP-samples/cap-sflight/blob/dfc7827da843ace0ea126f76fc78a6591b325c67/srv/travel-service.js#L13-L22). ::: ## Cache Control CAP provides the option to set a [Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control) header with a [max-age](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#max-age) directive to indicate that a response remains fresh until _n_ seconds after it was generated . In the CDS model, this can be done using the `@http.CacheControl: {maxAge: }` annotation on stream properties. The header indicates that caches can store the response and reuse it for subsequent requests while it's fresh. The `max-age` (in seconds) specifies the maximum age of the content before it becomes stale. :::info Elapsed time since the response was generated The `max-age` is the elapsed time since the response was generated on the origin server. It's not related to when the response was received. ::: ::: warning Only Java Cache Control feature is currently supported on the Java runtime only. :::

# Using Databases This guide provides instructions on how to use databases with CAP applications. Out of the box-support is provided for SAP HANA, SQLite, H2 (Java only), and PostgreSQL. ## Setup & Configuration

### Adding Database Packages {.node} Following are cds-plugin packages for CAP Node.js runtime that support respective databases: | Database | Package | Remarks | | ------------------------------ | ------------------------------------------------------------ | ---------------------------------- | | **[SAP HANA Cloud](databases-hana)** | [`@cap-js/hana`](https://www.npmjs.com/package/@cap-js/hana) | recommended for production | | **[SQLite](databases-sqlite)** | [`@cap-js/sqlite`](https://www.npmjs.com/package/@cap-js/sqlite) | recommended for development | | **[PostgreSQL](databases-postgres)** | [`@cap-js/postgres`](https://www.npmjs.com/package/@cap-js/postgres) | maintained by community + CAP team | > Follow the links above to find specific information for each. In general, all you need to do is to install one of the database packages, as follows: Using SQLite for development: ```sh npm add @cap-js/sqlite -D ``` Using SAP HANA for production: ```sh npm add @cap-js/hana ``` ::: details Prefer `cds add hana` ... ... which also does the equivalent of `npm add @cap-js/hana` but in addition cares for updating `mta.yaml` and other deployment resources as documented in the [deployment guide](deployment/to-cf#_1-using-sap-hana-database). ::: ### Auto-Wired Configuration {.node} The afore-mentioned packages use `cds-plugin` techniques to automatically configure the primary database with `cds.env`. For example, if you added SQLite and SAP HANA, this effectively results in this auto-wired configuration: ```json {"cds":{ "requires": { "db": { "[development]": { "kind": "sqlite", "impl": "@cap-js/sqlite", "credentials": { "url": "memory" } }, "[production]": { "kind": "hana", "impl": "@cap-js/hana", "deploy-format": "hdbtable" } } } }} ``` ::: details In contrast to pre-CDS 7 setups this means... 1. You don't need to — and should not — add direct dependencies to driver packages, like [`hdb`](https://www.npmjs.com/package/hdb) or [`sqlite3`](https://www.npmjs.com/package/sqlite3) anymore in your *package.json* files. 2. You don't need to configure `cds.requires.db` anymore, unless you want to override defaults brought with the new packages. ::: ### Custom Configuration {.node} The auto-wired configuration uses configuration presets, which are automatically enabled via `cds-plugin` techniques. You can always use the basic configuration and override individual properties to create a different setup: 1. Install a database driver package, for example: ```sh npm add @cap-js/sqlite ``` > Add option `-D` if you want this for development only. 2. Configure the primary database as a required service through `cds.requires.db`, for example: ```json {"cds":{ "requires": { "db": { "kind": "sqlite", "impl": "@cap-js/sqlite", "credentials": { "url": "db.sqlite" } } } }} ``` The config options are as follows: - `kind` — a name of a preset, like `sql`, `sqlite`, `postgres`, or `hana` - `impl` — the module name of a CAP database service implementation - `credentials` — an object with db-specific configurations, most commonly `url` ::: warning Don't configure credentials Credentials like `username` and `password` should **not** be added here but provided through service bindings, for example, via `cds bind`. ::: ::: tip Use `cds env` to inspect effective configuration For example, running this command: ```sh cds env cds.requires.db ``` → prints: ```sh { kind: 'sqlite', impl: '@cap-js/sqlite', credentials: { url: 'db.sqlite' } } ``` :::

### Built-in Database Support {.java} CAP Java has built-in support for different SQL-based databases via JDBC. This section describes the different databases and any differences between them with respect to CAP features. There's out of the box support for SAP HANA with CAP currently as well as H2 and SQLite. However, it's important to note that H2 and SQLite aren't enterprise grade databases and are recommended for non-productive use like local development or CI tests only. PostgreSQL is supported in addition, but has various limitations in comparison to SAP HANA, most notably in the area of schema evolution. Database support is enabled by adding a Maven dependency to the JDBC driver, as shown in the following table: | Database | JDBC Driver | Remarks | | ------------------------------ | ------------------------------------------------------------ | ---------------------------------- | | **[SAP HANA Cloud](databases-hana)** | `com.sap.cloud.db.jdbc:ngdbc` | Recommended for productive use | | **[H2](databases-h2)** | `com.h2database:h2` | Recommended for development and CI | | **[SQLite](databases-sqlite)** | `org.xerial:sqlite-jdbc` | Supported for development and CI
Recommended for local MTX | | **[PostgreSQL](databases-postgres)** | `org.postgresql:postgresql` | Supported for productive use | [Learn more about supported databases in CAP Java and their configuration](../java/cqn-services/persistence-services#database-support){ .learn-more} ## Providing Initial Data You can use CSV files to fill your database with initial data - see [Location of CSV Files](#location-of-csv-files).

For example, in our [*cap/samples/bookshop*](https://github.com/SAP-samples/cloud-cap-samples/tree/main/bookshop/db/data) application, we do so for *Books*, *Authors*, and *Genres* as follows: ```zsh bookshop/ ├─ db/ │ ├─ data/ # place your .csv files here │ │ ├─ sap.capire.bookshop-Authors.csv │ │ ├─ sap.capire.bookshop-Books.csv │ │ ├─ sap.capire.bookshop-Books.texts.csv │ │ └─ sap.capire.bookshop-Genres.csv │ └─ schema.cds └─ ... ```

For example, in our [CAP Samples for Java](https://github.com/SAP-samples/cloud-cap-samples-java/tree/main/db/data) application, we do so for some entities such as *Books*, *Authors*, and *Genres* as follows: ```zsh db/ ├─ data/ # place your .csv files here │ ├─ my.bookshop-Authors.csv │ ├─ my.bookshop-Books.csv │ ├─ my.bookshop-Books.texts.csv │ ├─ my.bookshop-Genres.csv │ └─ ... └─ index.cds ```

The **filenames** are expected to match fully qualified names of respective entity definitions in your CDS models, optionally using a dash `-` instead of a dot `.` for cosmetic reasons. ### Using `.csv` Files The **content** of these files is standard CSV content with the column titles corresponding to declared element names, like for `Books`: ::: code-group ```csvc [db/data/sap.capire.bookshop-Books.csv] ID,title,author_ID,stock 201,Wuthering Heights,101,12 207,Jane Eyre,107,11 251,The Raven,150,333 252,Eleonora,150,555 271,Catweazle,170,22 ``` ::: > Note: `author_ID` is the generated foreign key for the managed Association `author` → learn more about that in the [Generating SQL DDL](#generating-sql-ddl) section. If your content contains ... - commas or line breaks → enclose it in double quotes `"..."` - double quotes → escape them with doubled double quotes: `""...""` ```csvc ID,title,descr 252,Eleonora,"""Eleonora"" is a short story by Edgar Allan Poe, first published in 1842 in Philadelphia in the literary annual The Gift." ``` ::: danger On SAP HANA, only use CSV files for _configuration data_ that can't be changed by application users. → See [CSV data gets overridden in the SAP HANA guide for details](databases-hana#csv-data-gets-overridden). ::: ### Use `cds add data` Run this to generate an initial set of empty `.csv` files with header lines based on your CDS model: ```sh cds add data ``` ### Location of CSV Files CSV files can be found in the folders _db/data_ and _test/data_, as well as in any _data_ folder next to your CDS model files. When you use `cds watch` or `cds deploy`, CSV files are loaded by default from _test/data_. However, when preparing for production deployments using `cds build`, CSV files from _test/data_ are not loaded. ::: details Adding initial data next to your data model The content of these 'co-located' `.cds` files actually doesn't matter, but they need to be included in your data model, through a `using` clause in another file for example. If you need to use certain CSV files exclusively for your production deployments, but not for tests, you can achieve this by including them in a separate data folder, for example, _db/hana/data_. Create an _index.cds_ file in the _hana_ folder as outlined earlier. Then, set up this model location in a dummy cds service, for example _hanaDataSrv_, using the `[production]` profile. ```json "cds": { "requires": { "[production]": { "hanaDataSrv ": { "model": "hana" } } } } ```` As a consequence, when you run `cds build -–production` the model folder _hana_ is added, but it's not added when you run `cds deploy` or `cds watch` because the development profile is used by default. You can verify this by checking the cds build logs for the hana build task. Of course, this mechanism can also be used for PostgreSQL database deployments. ::: ::: details On SAP HANA ... CSV and _hdbtabledata_ files found in the _src_ folder of your database module are treated as native SAP HANA artifacts and deployed as they are. This approach offers the advantage of customizing the _hdbtabledata_ files if needed, such as adding a custom `include_filter` setting to mix initial and customer data in one table. However, the downside is that you must redundantly maintain them to keep them in sync with your CSV files. ::: Quite frequently you need to distinguish between sample data and real initial data. CAP supports this by allowing you to provide initial data in two places:

| Location | Deployed... | Purpose | | ----------- | -------------------- | -------------------------------------------------------- | | `db/data` | always | initial data for configurations, code lists, and similar | | `test/data` | if not in production | sample data for tests and demos |

Use the properties [cds.dataSource.csv.*](../java/developing-applications/properties#cds-dataSource-csv) to configure the location of the CSV files. You can configure different sets of CSV files in different [Spring profiles](https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.profiles). This configuration reads CSV data from `test/data` if the profile `test` is active: ::: code-group ```yaml [srv/src/main/resources/application.yaml] --- spring: config.activate.on-profile: test cds: dataSource.csv.paths: - test/data/** ``` :::

## Querying at Runtime Most queries to databases are constructed and executed from [generic event handlers of CRUD requests](providing-services#serving-crud), so quite frequently there's nothing to do. The following is for the remaining cases where you have to provide custom logic, and as part of it execute database queries. ### DB-Agnostic Queries

At runtime, we usually construct and execute queries using cds.ql APIs in a database-agnostic way. For example, queries like this are supported for all databases: ```js SELECT.from (Authors, a => { a.ID, a.name, a.books (b => { b.ID, b.title }) }) .where ({name:{like:'A%'}}) .orderBy ('name') ```

At runtime, we usually construct queries using the [CQL Query Builder API](../java/working-with-cql/query-api) in a database-agnostic way. For example, queries like this are supported for all databases: ```java Select.from(AUTHOR) .columns(a -> a.id(), a -> a.name(), a -> a.books().expand(b -> b.id(), b.title())) .where(a -> a.name().startWith("A")) .orderBy(a -> a.name()); ```

### Standard Operators {.node} The database services guarantee identical behavior of these operators: * `==`, `=` — with `=` null being translated to `is null` * `!=`, `<>` — with `!=` translated to `IS NOT` in SQLite, or to `IS DISTINCT FROM` in standard SQL, or to an equivalent polyfill in SAP HANA * `<`, `>`, `<=`, `>=`, `IN`, `LIKE` — are supported as is in standard SQL In particular, the translation of `!=` to `IS NOT` in SQLite — or to `IS DISTINCT FROM` in standard SQL, or to an equivalent polyfill in SAP HANA — greatly improves the portability of your code. ::: warning Runtime Only The operator mappings are available for runtime queries only, but not in CDS files. ::: ### Functions Mappings for Runtime Queries {.node} A specified set of standard functions is supported in a **database-agnostic**, hence portable way, and translated to database-specific variants or polyfills. Note that these functions are only supported within runtime queries, but not in CDS files. This set of functions are by large the same as specified in OData: * `concat(x,y,...)` — concatenates the given strings or numbers * `trim(x)` — removes leading and trailing whitespaces * `contains(x,y)` — checks whether `y` is contained in `x`, may be fuzzy * `startswith(x,y)` — checks whether `y` starts with `x` * `endswith(x,y)` — checks whether `y` ends with `x` * `matchespattern(x,y)` — checks whether `x` matches regex `y` * `substring(x,i,n?)` ¹ — Extracts a substring from `x` starting at index `i` (0-based) with optional length `n`. * **`i`**: Positive starts at `i`, negative starts `i` before the end. * **`n`**: Positive extracts `n` items; omitted extracts to the end; negative is invalid. * `indexof(x,y)` ¹ — returns the index of the first occurrence of `y` in `x` * `length(x)` — returns the length of string `x` * `tolower(x)` — returns all-lowercased `x` * `toupper(x)` — returns all-uppercased `x` * `ceiling(x)` — rounds the input numeric parameter up to the nearest numeric value * `floor(x)` — rounds the input numeric parameter down to the nearest numeric value * `round(x)` — rounds the input numeric parameter to the nearest numeric value. The mid-point between two integers is rounded away from zero, i.e. 0.5 is rounded to 1 and ‑0.5 is rounded to -1. * `year(x)` `month(x)`, `day(x)`, `hour(x)`, `minute(x)`, `second(x)` — returns parts of a datetime for a given `cds.DateTime` / `cds.Date` / `cds.Time` * `time(x)`, `date(x)` - returns a string representing the `time` / `date` for a given `cds.DateTime` / `cds.Date` / `cds.Time` * `fractionalseconds(x)` - returns a a `Decimal` representing the fractions of a second for a given `cds.Timestamp` * `maxdatetime()` - returns the latest possible point in time: `'9999-12-31T23:59:59.999Z'` * `mindatetime()` — returns the earliest possible point in time: `'0001-01-01T00:00:00.000Z'` * `totalseconds(x)` — returns the duration of the value in total seconds, including fractional seconds. The [OData spec](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part2-url-conventions.html#sec_totalseconds) defines the input as EDM.Duration: `P12DT23H59M59.999999999999S` * `now()` — returns the current datetime * `min(x)` `max(x)` `sum(x)` `average(x)` `count(x)`, `countdistinct(x)` — aggregate functions * `search(xs,y)` — checks whether `y` is contained in any of `xs`, may be fuzzy → [see Searching Data](../guides/providing-services#searching-data) * `session_context(v)` — with standard variable names → [see Session Variables](#session-variables) > ¹ These functions work zero-based. E.g., `substring('abcdef', 1, 3)` returns 'bcd' > You have to write these functions exactly as given; all-uppercase usages aren't supported. In addition to the standard functions, which all `@cap-js` database services support, `@cap-js/sqlite` and `@cap-js/postgres` also support these common SAP HANA functions, to further increase the scope for portable testing: * `years_between` — Computes the number of years between two specified dates. * `months_between` — Computes the number of months between two specified dates. * `days_between` — Computes the number of days between two specified dates. * `seconds_between` — Computes the number of seconds between two specified dates. * `nano100_between` — Computes the time difference between two dates to the precision of 0.1 microseconds. The database service implementation translates these to the best-possible native SQL functions, thus enhancing the extent of **portable** queries. With open source and the new database service architecture, we also have methods in place to enhance this list by custom implementation. > For the SAP HANA functions, both usages are allowed: all-lowercase as given above, as well as all-uppercase. ::: warning Runtime Only The function mappings are available for runtime queries only, but not in CDS files. ::: ### Session Variables {.node} The API shown below, which includes the function `session_context()` and specific pseudo variable names, is supported by **all** new database services, that is, *SQLite*, *PostgreSQL* and *SAP HANA*. This allows you to write respective code once and run it on all these databases: ```sql SELECT session_context('$user.id') SELECT session_context('$user.locale') SELECT session_context('$valid.from') SELECT session_context('$valid.to') ``` Among other things, this allows us to get rid of static helper views for localized data like `localized_de_sap_capire_Books`. ### Native DB Queries If required you can also use native database features by executing native SQL queries:

```js cds.db.run (`SELECT from sqlite_schema where name like ?`, name) ```

Use Spring's [JDBC Template](https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/jdbc/core/JdbcTemplate.html) to [leverage native database features](../java/cqn-services/persistence-services#jdbctemplate) as follows: ```java @Autowired JdbcTemplate db; ... db.queryForList("SELECT from sqlite_schema where name like ?", name); ```

### Reading `LargeBinary` / BLOB {.node} Formerly, `LargeBinary` elements (or BLOBs) were always returned as any other data type. Now, they are skipped from `SELECT *` queries. Yet, you can still enforce reading BLOBs by explicitly selecting them. Then the BLOB properties are returned as readable streams. ```js SELECT.from(Books) //> [{ ID, title, ..., image1, image2 }] // [!code --] SELECT.from(Books) //> [{ ID, title, ... }] SELECT(['image1', 'image2']).from(Books) //> [{ image1, image2 }] // [!code --] SELECT(['image1', 'image2']).from(Books) //> [{ image1: Readable, image2: Readable }] ``` [Read more about custom streaming in Node.js.](../node.js/best-practices#custom-streaming-beta){.learn-more} ## Generating DDL Files {#generating-sql-ddl}

When you run your server with `cds watch` during development, an in-memory database is bootstrapped automatically, with SQL DDL statements generated based on your CDS models. You can also do this manually with the CLI command `cds compile --to `.

When you've created a CAP Java application with `cds init --java` or with CAP Java's [Maven archetype](../java/developing-applications/building#the-maven-archetype), the Maven build invokes the CDS compiler to generate a `schema.sql` file for your target database. In the `default` profile (development mode), an in-memory database is [initialized by Spring](https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto.data-initialization) and the schema is bootstrapped from the `schema.sql` file. [Learn more about adding an inital database schema.](../java/cqn-services/persistence-services#initial-database-schema){.learn-more}

### Using `cds compile` For example, given these CDS models (derived from [*cap/samples/bookshop*](https://github.com/SAP-samples/cloud-cap-samples/tree/main/bookshop)): ::: code-group ```cds [db/schema.cds] using { Currency } from '@sap/cds/common'; namespace sap.capire.bookshop; entity Books { key ID : UUID; title : localized String; descr : localized String; author : Association to Authors; price : { amount : Decimal; currency : Currency; } } entity Authors { key ID : UUID; name : String; books : Association to many Books on books.author = $self; } ``` ::: ::: code-group ```cds [srv/cat-service.cds] using { sap.capire.bookshop as my } from '../db/schema'; service CatalogService { entity ListOfBooks as projection on Books { *, author.name as author } } ``` ::: Generate an SQL DDL script by running this in the root directory containing both *.cds* files:

```sh cds compile srv/cat-service --to sql --dialect sqlite > schema.sql ``` Output: ::: code-group ```sql [schema.sql] CREATE TABLE sap_capire_bookshop_Books ( ID NVARCHAR(36) NOT NULL, title NVARCHAR(5000), descr NVARCHAR(5000), author_ID NVARCHAR(36), price_amount DECIMAL, price_currency_code NVARCHAR(3), PRIMARY KEY(ID) ); CREATE TABLE sap_capire_bookshop_Authors ( ID NVARCHAR(36) NOT NULL, name NVARCHAR(5000), PRIMARY KEY(ID) ); CREATE TABLE sap_common_Currencies ( name NVARCHAR(255), descr NVARCHAR(1000), code NVARCHAR(3) NOT NULL, symbol NVARCHAR(5), minorUnit SMALLINT, PRIMARY KEY(code) ); CREATE TABLE sap_capire_bookshop_Books_texts ( locale NVARCHAR(14) NOT NULL, ID NVARCHAR(36) NOT NULL, title NVARCHAR(5000), descr NVARCHAR(5000), PRIMARY KEY(locale, ID) ); CREATE VIEW CatalogService_ListOfBooks AS SELECT Books.ID, Books.title, Books.descr, author.name AS author, Books.price_amount, Books.price_currency_code FROM sap_capire_bookshop_Books AS Books LEFT JOIN sap_capire_bookshop_Authors AS author ON Books.author_ID = author.ID; --- some more technical views skipped ... ``` :::

```sh cds compile srv/cat-service --to sql > schema.sql ``` Output: ::: code-group ```sql [schema.sql] CREATE TABLE sap_capire_bookshop_Books ( createdAt TIMESTAMP(7), createdBy NVARCHAR(255), modifiedAt TIMESTAMP(7), modifiedBy NVARCHAR(255), ID INTEGER NOT NULL, title NVARCHAR(111), descr NVARCHAR(1111), author_ID INTEGER, genre_ID INTEGER, stock INTEGER, price DECFLOAT, currency_code NVARCHAR(3), image BINARY LARGE OBJECT, PRIMARY KEY(ID) ); CREATE TABLE sap_capire_bookshop_Books ( ID NVARCHAR(36) NOT NULL, title NVARCHAR(5000), descr NVARCHAR(5000), author_ID NVARCHAR(36), price_amount DECIMAL, price_currency_code NVARCHAR(3), PRIMARY KEY(ID) ); CREATE TABLE sap_capire_bookshop_Authors ( ID NVARCHAR(36) NOT NULL, name NVARCHAR(5000), PRIMARY KEY(ID) ); CREATE TABLE sap_common_Currencies ( name NVARCHAR(255), descr NVARCHAR(1000), code NVARCHAR(3) NOT NULL, symbol NVARCHAR(5), minorUnit SMALLINT, PRIMARY KEY(code) ); CREATE TABLE sap_capire_bookshop_Books_texts ( locale NVARCHAR(14) NOT NULL, ID NVARCHAR(36) NOT NULL, title NVARCHAR(5000), descr NVARCHAR(5000), PRIMARY KEY(locale, ID) ); CREATE VIEW CatalogService_ListOfBooks AS SELECT Books_0.createdAt, Books_0.modifiedAt, Books_0.ID, Books_0.title, Books_0.author, Books_0.genre_ID, Books_0.stock, Books_0.price, Books_0.currency_code, Books_0.image FROM CatalogService_Books AS Books_0; CREATE VIEW CatalogService_ListOfBooks AS SELECT Books.ID, Books.title, Books.descr, author.name AS author, Books.price_amount, Books.price_currency_code FROM sap_capire_bookshop_Books AS Books LEFT JOIN sap_capire_bookshop_Authors AS author ON Books.author_ID = author.ID; --- some more technical views skipped ... ``` :::

::: tip Use the specific SQL dialect (`hana`, `sqlite`, `h2`, `postgres`) with `cds compile --to sql --dialect ` to get DDL that matches the target database. ::: ### Rules for Generated DDL A few observations on the generated SQL DDL output: 1. **Tables / Views** — Declared entities become tables, projected entities become views. 2. **Type Mapping** — [CDS types are mapped to database-specific SQL types](../cds/types). 3. **Slugified FQNs** — Dots in fully qualified CDS names become underscores in SQL names. 4. **Flattened Structs** — Structured elements like `Books:price` are flattened with underscores. 5. **Generated Foreign Keys** — For managed to-one Associations, foreign key columns are created. For example, this applies to `Books:author`. In addition, you can use the following annotations to fine-tune generated SQL. ### @cds.persistence.skip Add `@cds.persistence.skip` to an entity to indicate that this entity should be skipped from generated DDL scripts, and also no SQL views to be generated on top of it: ```cds @cds.persistence.skip entity Foo {...} //> No SQL table will be generated entity Bar as select from Foo; //> No SQL view will be generated ``` ### @cds.persistence.exists Add `@cds.persistence.exists` to an entity to indicate that this entity should be skipped from generated DDL scripts. In contrast to `@cds.persistence.skip` a database relation is expected to exist, so we can generate SQL views on top. ```cds @cds.persistence.exists entity Foo {...} //> No SQL table will be generated entity Bar as select from Foo; //> The SQL view will be generated ``` ::: details On SAP HANA ... If the respective entity is a user-defined function or a calculation view, one of the annotations `@cds.persistence.udf` or `@cds.persistence.calcview` also needs to be assigned. See [Calculated Views and User-Defined Functions](../advanced/hana#calculated-views-and-user-defined-functions) for more details. ::: ### @cds.persistence.table Annotate an entity with `@cds.persistence.table` to create a table with the effective signature of the view definition instead of an SQL view. ```cds @cds.persistence.table entity Foo as projection on Bar {...} ``` > All parts of the view definition not relevant for the signature (like `where`, `group by`, ...) are ignored. Use case for this annotation: Use projections on imported APIs as replica cache tables. ### @sql.prepend / append Use `@sql.prepend` and `@sql.append` to add native SQL clauses to before or after generated SQL output of CDS entities or elements. Example: ````cds @sql.append: ```sql GROUP TYPE foo GROUP SUBTYPE bar ``` entity E { ..., @sql.append: 'FUZZY SEARCH INDEX ON' text: String(100); } @sql.append: 'WITH DDL ONLY' entity V as select from E { ... }; ```` Output: ```sql CREATE TABLE E ( ..., text NVARCHAR(100) FUZZY SEARCH INDEX ON ) GROUP TYPE foo GROUP SUBTYPE bar; CREATE VIEW V AS SELECT ... FROM E WITH DDL ONLY; ``` The following rules apply: - The value of the annotation must be a [string literal](../cds/cdl#multiline-literals). - The compiler doesn't check or process the provided SQL snippets in any way. You're responsible to ensure that the resulting statement is valid and doesn't negatively impact your database or your application. We don't provide support for problems caused by using this feature. - If you refer to a column name in the annotation, you need to take care of a potential name mapping yourself, for example, for structured elements. - Annotation `@sql.prepend` is only supported for entities translating to tables. It can't be used with views nor with elements. - For SAP HANA tables, there's an implicit `@sql.prepend: 'COLUMN'` that is overwritten by an explicitly provided `@sql.prepend`. * Both `@sql.prepend` and `@sql.append` are disallowed in SaaS extension projects. If you use native database clauses in combination with `@cds.persistence.journal`, see [Schema Evolution Support of Native Database Clauses](databases-hana#schema-evolution-native-db-clauses). #### Creating a Row Table on SAP HANA By using `@sql.prepend: 'ROW'`, you can create a row table: ```cds @sql.prepend: 'ROW' entity E { key id: Integer; } ``` Run `cds compile - 2 hdbtable` on the previous sample and this is the result: ```sql [E.hdbtable] ROW TABLE E ( id INTEGER NOT NULL, PRIMARY KEY(id) ) ``` [Learn more about Columnar and Row-Based Data Storage](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-administration-guide/columnar-and-row-based-data-storage){.learn-more} ### Reserved Words The CDS compiler and CAP runtimes provide smart quoting for reserved words in SQLite and in SAP HANA so that they can still be used in most situations. But in general reserved words cannot be used as identifiers. The list of reserved words varies per database. Find here a collection of resources on selected databases and their reference documentation: * [SAP HANA SQL Reference Guide for SAP HANA Platform (Cloud Version)](https://help.sap.com/docs/HANA_SERVICE_CF/7c78579ce9b14a669c1f3295b0d8ca16/28bcd6af3eb6437892719f7c27a8a285.html) * [SAP HANA SQL Reference Guide for SAP HANA Cloud](https://help.sap.com/docs/HANA_CLOUD_DATABASE/c1d3f60099654ecfb3fe36ac93c121bb/28bcd6af3eb6437892719f7c27a8a285.html) * [SQLite Keywords](https://www.sqlite.org/lang_keywords.html) * [H2 Keywords/Reserved Words](https://www.h2database.com/html/advanced.html#keywords) * [PostgreSQL SQL Key Words](https://www.postgresql.org/docs/current/sql-keywords-appendix.html) [There are also reserved words related to SAP Fiori.](../advanced/fiori#reserved-words){.learn-more} ## Database Constraints The information about foreign key relations contained in the associations of CDS models can be used to generate foreign key constraints on the database tables. Within CAP, referential consistency is established only at commit. The ["deferred" concept for foreign key constraints](https://www.sqlite.org/foreignkeys.html) in SQL databases allows the constraints to be checked and enforced at the time of the [COMMIT statement within a transaction](https://www.sqlite.org/lang_transaction.html) rather than immediately when the data is modified, providing more flexibility in maintaining data integrity. Enable generation of foreign key constraints on the database with: cds.features.assert_integrity = db ::: warning Database constraints are not supported for H2 Referential constraints on H2 cannot be defined as "deferred", which is needed for database constraints within CAP. ::: With that switched on, foreign key constraints are generated for managed to-one associations. For example, given this model: ```cds entity Books { key ID : Integer; ... author : Association to Authors; } entity Authors { key ID : Integer; ... } ``` The following `Books_author` constraint would be added to table `Books`: ```sql CREATE TABLE Authors ( ID INTEGER NOT NULL, -- primary key referenced by the constraint ..., PRIMARY KEY(ID) ); CREATE TABLE Books ( ID INTEGER NOT NULL, author_ID INTEGER, -- generated foreign key field ..., PRIMARY KEY(ID), CONSTRAINT Books_author -- constraint is explicitly named // [!code focus] FOREIGN KEY(author_ID) -- link generated foreign key field author_ID ... REFERENCES Authors(ID) -- ... with primary key field ID of table Authors ON UPDATE RESTRICT ON DELETE RESTRICT VALIDATED -- validate existing entries when constraint is created ENFORCED -- validate changes by insert/update/delete INITIALLY DEFERRED -- validate only at commit ) ``` No constraints are generated for... * Unmanaged associations or compositions * To-many associations or compositions * Associations annotated with `@assert.integrity: false` * Associations where the source or target entity is annotated with `@cds.persistence.exists` or `@cds.persistence.skip` If the association is the backlink of a **composition**, the constraint's delete rule changes to `CASCADE`. That applies, for example, to the `parent` association in here: ```cds entity Genres { key ID : Integer; parent : Association to Genres; children : Composition of many Genres on children.parent = $self; } ``` As a special case, a referential constraint with `delete cascade` is also generated for the text table of a [localized entity](../guides/localized-data#localized-data), although no managed association is present in the `texts` entity. Add a localized element to entity `Books` from the previous example: ```cds entity Books { key ID : Integer; ... title : localized String; } ``` The generated text table then is: ```sql CREATE TABLE Books_texts ( locale NVARCHAR(14) NOT NULL, ID INTEGER NOT NULL, title NVARCHAR(5000), PRIMARY KEY(locale, ID), CONSTRAINT Books_texts_texts // [!code focus] FOREIGN KEY(ID) REFERENCES Books(ID) ON UPDATE RESTRICT ON DELETE CASCADE VALIDATED ENFORCED INITIALLY DEFERRED ) ``` ::: warning Database constraints aren't intended for checking user input Instead, they protect the integrity of your data in the database layer against programming errors. If a constraint violation occurs, the error messages coming from the database aren't standardized by the runtimes but presented as-is. → Use [`@assert.target`](providing-services#assert-target) for corresponding input validations. ::: ## Using Native Features { #native-db-functions} In general, the CDS 2 SQL compiler doesn't 'understand' SQL functions but translates them to SQL generically as long as they follow the standard call syntax of `function(param1, param2)`. This allows you to use native database functions inside your CDS models. Example: ```cds entity BookPreview as select from Books { IFNULL (descr, title) as shorttext //> using HANA function IFNULL }; ``` The `OVER` clause for SQL Window Functions is supported, too: ```cds entity RankedBooks as select from Books { name, author, rank() over (partition by author order by price) as rank }; ``` #### Using Native Functions with Different DBs { #sqlite-and-hana-functions} In case of conflicts, follow these steps to provide different models for different databases: 1. Add database-specific schema extensions in specific subfolders of `./db`: ::: code-group ```cds [db/sqlite/index.cds] using { AdminService } from '..'; extend projection AdminService.Authors with { strftime('%Y',dateOfDeath)-strftime('%Y',dateOfBirth) as age : Integer } ``` ```cds [db/hana/index.cds] using { AdminService } from '..'; extend projection AdminService.Authors with { YEARS_BETWEEN(dateOfBirth, dateOfDeath) as age : Integer } ``` ::: 2. Add configuration in specific profiles to your *package.json*, to use these database-specific extensions: ```json { "cds": { "requires": { "db": { "kind": "sql", "[development]": { "model": "db/sqlite" }, "[production]": { "model": "db/hana" } } }}} ```

:::info The following steps are only needed when you use two different local databases. 3. For CAP Java setups you might need to reflect the different profiles in your CDS Maven plugin configuration. This might not be needed for all setups, like using a standard local database (sqlite, H2, or PostgreSQL) and a production SAP HANA setup. In that case the local build defaults to the `development` profile. But for other setups, like using a local PostgreSQL and a local SQLite you'll need two (profiled) `cds deploy` commands: ```xml cds.build cds build --for java deploy --profile development --dry --out "${project.basedir}/src/main/resources/schema-h2.sql" deploy --profile production --dry --out "${project.basedir}/src/main/resources/schema-postresql.sql" ``` 4. For the Spring Boot side it's similar. If you have a local development database and a hybrid profile with a remote SAP HANA database, you only need to run in default (or any other) profile. For the SAP HANA part, the build and deploy part is done separately and the application just needs to be started using `cds bind`. Once you have 2 non-HANA local databases you need to have 2 distinct database configurations in your Spring Boot configuration (in most cases application.yaml). ```yaml spring: config: activate: on-profile: default,h2 sql: init: schema-locations: classpath:schema-h2.sql --- spring: config: activate: on-profile: postgresql sql: init: schema-locations: classpath:schema-postgresql.sql datasource: url: "jdbc:postgresql://localhost:5432/my_schema" driver-class-name: org.postgresql.Driver hikari: maximum-pool-size: 1 max-lifetime: 0 ``` In case you use 2 different databases you also need to make sure that you have the JDBC drivers configured (on the classpath). :::

CAP samples demonstrate this in [cap/samples/fiori](https://github.com/SAP-samples/cloud-cap-samples/commit/65c8c82f745e0097fab6ca8164a2ede8400da803).
There's also a [code tour](https://github.com/SAP-samples/cloud-cap-samples#code-tours) available for that. # Using SQLite for Development {#sqlite} CAP provides extensive support for [SQLite](https://www.sqlite.org/index.html), which allows projects to speed up development by magnitudes at minimized costs. We strongly recommend using this option as much as possible during development and testing.

::: tip New SQLite Service This guide focuses on the new SQLite Service provided through *[@cap-js/sqlite](https://www.npmjs.com/package/@cap-js/sqlite)*, which has many advantages over the former one, as documented in the [*Features*](#features) section. To migrate from the old service, find instructions in the [*Migration*](#migration) section. :::

[Learn more about the features and limitations of using CAP with SQlite.](../java/cqn-services/persistence-services#sqlite){.learn-more}

## Setup & Configuration

Run this to use SQLite for development: ```sh npm add @cap-js/sqlite -D ``` ### Auto-Wired Configuration {.node} The `@cap-js/sqlite` package uses the `cds-plugin` technique to auto-configure your application for using an in-memory SQLite database for development. You can inspect the effective configuration using `cds env`: ```sh cds env requires.db ``` Output: ```js { impl: '@cap-js/sqlite', credentials: { url: ':memory:' }, kind: 'sqlite' } ``` [See also the general information on installing database packages.](databases#setup-configuration){.learn-more}

### Using the Maven Archetype {.java} When a new CAP Java project is created with the [Maven Archetype](../java/developing-applications/building#the-maven-archetype), you can specify the in-memory database to be used. Use the option `-DinMemoryDatabase=sqlite` to create a project that uses SQLite as in-memory database. ### Manual Configuration {.java} To use SQLite, add a Maven dependency to the SQLite JDBC driver: ```xml org.xerial sqlite-jdbc runtime ``` Further configuration depends on whether you run SQLite as an [in-memory database](#in-memory-databases) or as a [file-based](#persistent-databases) database. ## Deployment

### Initial Database Schema Configure the build to create an initial _schema.sql_ file for SQLite using `cds deploy --to sqlite --dry --out srv/src/main/resources/schema.sql`. ::: code-group ```xml [srv/pom.xml] schema.sql cds deploy --to sqlite --dry --out srv/src/main/resources/schema.sql ``` ::: [Learn more about creating an initial database schema](/java/cqn-services/persistence-services#initial-database-schema-1){.learn-more}

### In-Memory Databases

As stated previously, `@cap-js/sqlite` uses an in-memory SQLite database by default. For example, when starting your application with `cds watch`, you can see this in the log output: ```log ... [cds] - connect to db > sqlite { url: ':memory:' } // [!code focus] > init from db/init.js > init from db/data/sap.capire.bookshop-Authors.csv > init from db/data/sap.capire.bookshop-Books.csv > init from db/data/sap.capire.bookshop-Books.texts.csv > init from db/data/sap.capire.bookshop-Genres.csv /> successfully deployed to in-memory database. // [!code focus] ... ``` ::: tip Using in-memory databases is the most recommended option for test drives and test pipelines. :::

The database content is stored in-memory. Configure the DB connection in the non-productive `default` profile: ::: code-group ```yaml [srv/src/main/resources/application.yaml] --- spring: config.activate.on-profile: default sql: init: mode: always datasource: url: "jdbc:sqlite:file::memory:?cache=shared" driver-class-name: org.sqlite.JDBC hikari: maximum-pool-size: 1 max-lifetime: 0 ``` ::: [Learn how to configure an in-memory SQLite database.](../java/cqn-services/persistence-services#in-memory-storage){.learn-more}

### Persistent Databases

You can also use persistent SQLite databases. Follow these steps to do so:

You can also use persistent SQLite databases. In this case, the schema is initialized by `cds deploy` and not by Spring. Follow these steps:

1. Specify a database filename in your `db` configuration as follows: ::: code-group ```json [package.json] { "cds": { "requires": { "db": { "kind": "sqlite", "credentials": { "url": "db.sqlite" } // [!code focus] } }}} ``` ::: 2. Run `cds deploy`: ```sh cds deploy ``` This will: 1. Create a database file with the given name. 2. Create the tables and views according to your CDS model. 3. Fill in initial data from the provided _.csv_ files.

With that in place, the server will use this prepared database instead of bootstrapping an in-memory one upon startup: ```log ... [cds] - connect to db > sqlite { url: 'db.sqlite' } ... ```

Finally, configure the DB connection - ideally in a dedicated `sqlite` profile: ::: code-group ```yaml [srv/src/main/resources/application.yaml] --- spring: config.activate.on-profile: sqlite datasource: url: "jdbc:sqlite:sqlite.db" driver-class-name: org.sqlite.JDBC hikari: maximum-pool-size: 1 ``` ::: [Learn how to configure a file-based SQLite database](../java/cqn-services/persistence-services#file-based-storage){.learn-more}

::: tip Redeploy on changes Remember to always redeploy your database whenever you change your models or your data. Just run `cds deploy` again to do so. ::: ### Drop-Create Schema When you redeploy your database, it will always drop-create all tables and views. This is **most suitable for development environments**, where schema changes are very frequent and broad. ### Schema Evolution While drop-create is most appropriate for development, it isn't suitable for database upgrades in production, as all customer data would be lost. To avoid this, `cds deploy` also supports automatic schema evolution, which you can use as follows: 1. Enable automatic schema evolution in your `db` configuration: ::: code-group ```json [package.json] { "cds": { "requires": { "db": { "kind": "sqlite", "credentials": { "url": "db.sqlite" }, "schema_evolution": "auto" // [!code focus] } }}} ``` ::: 2. Run `cds deploy`: ```sh cds deploy ``` [Learn more about automatic schema evolution in the PostgreSQL guide.
The information in there is also applicable to SQLite with persistent databases.](databases-postgres#schema-evolution) {.learn-more} ## Features

CAP supports most of the major features on SQLite: * [Path Expressions](../java/working-with-cql/query-api#path-expressions) & Filters * [Expands](../java/working-with-cql/query-api#projections) * [Localized Queries](../guides/localized-data#read-operations) * [Comparison Operators](../java/working-with-cql/query-api#comparison-operators) * [Predicate Functions](../java/working-with-cql/query-api#predicate-functions) [Learn about features and limitations of SQLite.](../java/cqn-services/persistence-services#sqlite){.learn-more}

The following is an overview of advanced features supported by the new database services. > These apply to all new database services, including SQLiteService, HANAService, and PostgresService. ### Path Expressions & Filters {.node} The new database service provides **full support** for all kinds of [path expressions](../cds/cql#path-expressions), including [infix filters](../cds/cql#with-infix-filters) and [exists predicates](../cds/cql#exists-predicate). For example, you can try this out with *[cap/samples](https://github.com/sap-samples/cloud-cap-samples)* as follows: ```js // $ cds repl --profile better-sqlite var { server } = await cds.test('bookshop'), { Books, Authors } = cds.entities await INSERT.into (Books) .entries ({ title: 'Unwritten Book' }) await INSERT.into (Authors) .entries ({ name: 'Upcoming Author' }) await SELECT `from ${Books} { title as book, author.name as author, genre.name as genre }` await SELECT `from ${Authors} { books.title as book, name as author, books.genre.name as genre }` await SELECT `from ${Books} { title as book, author[ID<170].name as author, genre.name as genre }` await SELECT `from ${Books} { title as book, author.name as author, genre.name as genre }` .where ({'author.name':{like:'Ed%'},or:{'author.ID':170}}) await SELECT `from ${Books} { title as book, author.name as author, genre.name as genre } where author.name like 'Ed%' or author.ID=170` await SELECT `from ${Books}:author[name like 'Ed%' or ID=170] { books.title as book, name as author, books.genre.name as genre }` await SELECT `from ${Books}:author[150] { books.title as book, name as author, books.genre.name as genre }` await SELECT `from ${Authors} { ID, name, books { ID, title }}` await SELECT `from ${Authors} { ID, name, books { ID, title, genre { ID, name }}}` await SELECT `from ${Authors} { ID, name, books.genre { ID, name }}` await SELECT `from ${Authors} { ID, name, books as some_books { ID, title, genre.name as genre }}` await SELECT `from ${Authors} { ID, name, books[genre.ID=11] as dramatic_books { ID, title, genre.name as genre }}` await SELECT `from ${Authors} { ID, name, books.genre[name!='Drama'] as no_drama_books_count { count(*) as sum }}` await SELECT `from ${Authors} { books.genre.ID }` await SELECT `from ${Authors} { books.genre }` await SELECT `from ${Authors} { books.genre.name }` ``` ### Optimized Expands {.node} The old database service implementation(s) used to translate deep reads, that is, SELECTs with expands, into several database queries and collect the individual results into deep result structures. The new service uses `json_object` and other similar functions to instead do that in one single query, with sub selects, which greatly improves performance. For example: ```sql SELECT.from(Authors, a => { a.ID, a.name, a.books (b => { b.title, b.genre (g => { g.name }) }) }) ``` While this used to require three queries with three roundtrips to the database, now only one query is required. ### Localized Queries {.node} With the old implementation, running queries like `SELECT.from(Books)` would always return localized data, without being able to easily read the non-localized data. The new service does only what you asked for, offering new `SELECT.localized` options: ```js let books = await SELECT.from(Books) //> non-localized data let lbooks = await SELECT.localized(Books) //> localized data ``` Usage variants include: ```js SELECT.localized(Books) SELECT.from.localized(Books) SELECT.one.localized(Books) ``` ### Using Lean Draft {.node} The old implementation was overly polluted with draft handling. But as draft is actually a Fiori UI concept, none of that should show up in database layers. Hence, we eliminated all draft handling from the new database service implementations, and implemented draft in a modular, non-intrusive way — called *'Lean Draft'*. The most important change is that we don't do expensive UNIONs anymore but work with single (cheap) selects. ### Consistent Timestamps {.node} Values for elements of type `DateTime` and `Timestamp` are handled in a consistent way across all new database services along these lines: :::tip *Timestamps* = `Timestamp` as well as `DateTime` When we say *Timestamps*, we mean elements of type `Timestamp` as well as `DateTime`. Although they have different precision levels, they are essentially the same type. `DateTime` elements have seconds precision, while `Timestamp` elements have milliseconds precision in SQLite, and microsecond precision in SAP HANA and PostgreSQL. ::: #### Writing Timestamps When writing data using INSERT, UPSERT or UPDATE, you can provide values for `DateTime` and `Timestamp` elements as JavaScript `Date` objects or ISO 8601 Strings. All input is normalized to ensure `DateTime` and `Timestamp` values can be safely compared. In case of SAP HANA and PostgreSQL, they're converted to native types. In case of SQLite, they're stored as ISO 8601 Strings in Zulu timezone as returned by JavaScript's `Date.toISOString()`. For example: ```js await INSERT.into(Books).entries([ { createdAt: new Date }, //> stored .toISOString() { createdAt: '2022-11-11T11:11:11Z' }, //> padded with .000Z { createdAt: '2022-11-11T11:11:11.123Z' }, //> stored as is { createdAt: '2022-11-11T11:11:11.1234563Z' }, //> truncated to .123Z { createdAt: '2022-11-11T11:11:11+02:00' }, //> converted to zulu time ]) ``` #### Reading Timestamps Timestamps are returned as they're stored in a normalized way, with milliseconds precision, as supported by the JavaScript `Date` object. For example, the entries inserted previously would return the following: ```js await SELECT('createdAt').from(Books).where({title:null}) ``` ```js [ { createdAt: '2023-08-10T14:24:30.798Z' }, { createdAt: '2022-11-11T11:11:11.000Z' }, { createdAt: '2022-11-11T11:11:11.123Z' }, { createdAt: '2022-11-11T11:11:11.123Z' }, { createdAt: '2022-11-11T09:11:11.000Z' } ] ``` `DateTime` elements are returned with seconds precision, with all fractional second digits truncated. That is, if the `createdAt` in our examples was a `DateTime`, the previous query would return this: ```js [ { createdAt: '2023-08-10T14:24:30Z' }, { createdAt: '2022-11-11T11:11:11Z' }, { createdAt: '2022-11-11T11:11:11Z' }, { createdAt: '2022-11-11T11:11:11Z' }, { createdAt: '2022-11-11T09:11:11Z' } ] ``` #### Comparing DateTimes & Timestamps You can safely compare DateTimes & Timestamps with each other and with input values. The input values have to be `Date` objects or ISO 8601 Strings in Zulu timezone with three fractional digits. For example, all of these would work: ```js SELECT.from(Foo).where `someTimestamp = anotherTimestamp` SELECT.from(Foo).where `someTimestamp = someDateTime` SELECT.from(Foo).where `someTimestamp = ${new Date}` SELECT.from(Foo).where `someTimestamp = ${req.timestamp}` SELECT.from(Foo).where `someTimestamp = ${'2022-11-11T11:11:11.123Z'}` ``` While these would fail, because the input values don't comply to the rules: ```js SELECT.from(Foo).where `createdAt = ${'2022-11-11T11:11:11+02:00'}` // non-Zulu time zone SELECT.from(Foo).where `createdAt = ${'2022-11-11T11:11:11Z'}` // missing 3-digit fractions ``` > This is because we can never reliably infer the types of input to `where` clause expressions. Therefore, that input will not receive any normalisation, but be passed down as is as plain string. :::tip Always ensure proper input in `where` clauses Either use strings strictly in `YYYY-MM-DDThh:mm:ss.fffZ` format, or `Date` objects, as follows: ```js SELECT.from(Foo).where ({ createdAt: '2022-11-11T11:11:11.000Z' }) SELECT.from(Foo).where ({ createdAt: new Date('2022-11-11T11:11:11Z') }) ``` ::: The rules regarding Timestamps apply to all comparison operators: `=`, `<`, `>`, `<=`, `>=`. ### Improved Performance {.node} The combination of the above-mentioned improvements commonly leads to significant performance improvements. For example, displaying the list page of Travels in [cap/sflight](https://github.com/SAP-samples/cap-sflight) took **>250ms** in the past, and **~15ms** now. ## Migration {.node} While we were able to keep all public APIs stable, we had to apply changes and fixes to some **undocumented behaviours and internal APIs** in the new implementation. While not formally breaking changes, you may have used or relied on these undocumented APIs and behaviours. In that case, you can find instructions about how to resolve this in the following sections. > These apply to all new database services: SQLiteService, HANAService, and PostgresService. ### Use Old and New in Parallel {.node} During migration, you may want to occasionally run and test your app with both the new SQLite service and the old one. You can accomplish this as follows: 1. Add the new service with `--no-save`: ```sh npm add @cap-js/sqlite --no-save ``` > This bypasses the *cds-plugin* mechanism, which works through package dependencies. 2. Run or test your app with the `better-sqlite` profile using one of these options: ```sh cds watch bookshop --profile better-sqlite ``` ```sh CDS_ENV=better-sqlite cds watch bookshop ``` ```sh CDS_ENV=better-sqlite jest --silent ``` 3. Run or test your app with the old SQLite service as before: ```sh cds watch bookshop ``` ```sh jest --silent ``` ### Avoid UNIONs and JOINs {.node} Many advanced features supported by the new database services, like path expressions or deep expands, rely on the ability to infer queries from CDS models. This task gets extremely complex when adding UNIONs and JOINs to the equation — at least the effort and overhead is hardly matched by generated value. Therefore, we dropped support of UNIONs and JOINs in CQN queries. For example, this means queries like these are deprecated / not supported any longer: ```js SELECT.from(Books).join(Authors,...) ``` Mitigations: 1. Use [path expressions](#path-expressions-filters) instead of joins. (The former lack of support for path expressions was the most common reason for having to use joins at all.) 2. Use plain SQL queries like so: ```js await db.run(`SELECT from ${Books} join ${Authors} ...`) ``` 3. Use helper views modeled in CDS, which still supports all complex UNIONs and JOINs, then use this view via `cds.ql`. ### Fixed Localized Data {.node} Formerly, when reading data using `cds.ql`, this *always* returned localized data. For example: ```js SELECT.from(Books) // always read from localized.Books instead ``` This wasn't only wrong, but also expensive. Localized data is an application layer concept. Database services should return what was asked for, and nothing else. → Use [*Localized Queries*](#localized-queries) if you really want to read localized data from the database: ```js SELECT.localized(Books) // reads localized data SELECT.from(Books) // reads plain data ``` ::: details No changes to app services behaviour Generic application service handlers use *SELECT.localized* to request localized data from the database. Hence, CAP services automatically serve localized data as before. ::: ### Skipped Virtuals {.node} In contrast to their former behaviour, new database services ignore all virtual elements and hence don't add them to result set entries. Selecting only virtual elements in a query leads to an error. ::: details Reasoning Virtual elements are meant to be calculated and filled in by custom handlers of your application services. Nevertheless, the old database services always returned `null`, or specified `default` values for virtual elements. This behavior was removed, as it provides very little value, if at all. ::: For example, given this definition: ```cds entity Foo { foo : Integer; virtual bar : Integer; } ``` The behaviour has changed to: ```js [dev] cds repl > SELECT.from('Foo') //> [{ foo:1, bar:null }, ...] // [!code --] > SELECT.from('Foo') //> [{ foo:1 }, ...] > SELECT('bar').from('Foo') //> ERROR: no columns to read ``` ### <> Operator {.node} Before, both `<>` and `!=` were translated to `name <> 'John' OR name is null`. * The operator `<>` now works as specified in the SQL standard. * `name != 'John'` is translated as before to `name <> 'John' OR name is null`. ::: warning This is a breaking change in regard to the previous implementation. ::: ### Miscellaneous {.node} - Only `$now` and `$user` are supported as values for `@cds.on.insert/update`. - CQNs with subqueries require table aliases to refer to elements of outer queries. - Table aliases must not contain dots. - CQNs with an empty columns array now throw an error. - `*` isn't a column reference. Use `columns: ['*']` instead of `columns: [{ref:'*'}]`. - Column names in CSVs must map to physical column names: ```csvc ID;title;author_ID;currency_code // [!code ++] ID;title;author.ID;currency.code // [!code --] ``` ### Adopt Lean Draft {.node} As mentioned in [Using Lean Draft](#using-lean-draft), we eliminated all draft handling from new database service implementations, and instead implemented draft in a modular, non-intrusive, and optimized way — called *'Lean Draft'*. When using the new service, the new `cds.fiori.lean_draft` mode is automatically switched on. You may additionally switch on cds.fiori.draft_compat:true in case you run into problems. More detailed documentation for that is coming. ### Finalizing Migration {.node} When you have finished migration, remove the old [*sqlite3* driver](https://www.npmjs.com/package/sqlite3) : ```sh npm rm sqlite3 ``` And activate the new one as cds-plugin: ```sh npm add @cap-js/sqlite --save ```

## SQLite in Production? As stated in the beginning, SQLite is mostly intended to speed up development, but is not fit for production. This is not because of limited warranties or lack of support, but rather because of suitability. A major criterion is this: cloud applications are usually served by server clusters, in which each server is connected to a shared database. SQLite could only be used in such setups with the persistent database file accessed through a network file system. This is rarely available and results in slow performance. Hence, an enterprise client-server database is a more fitting choice for these scenarios. Having said this, there can indeed be scenarios where SQLite might also be used in production, such as using SQLite as in-memory caches. → [Find a detailed list of criteria on the sqlite.org website](https://www.sqlite.org/whentouse.html). ::: warning SQLite only has limited support for concurrent database access due to its very coarse lock granularity. This makes it badly suited for applications with high concurrency. ::: # Using H2 for Development in CAP Java For local development and testing, CAP Java supports the [H2](https://www.h2database.com/) database, which can be configured to run in-memory. [Learn more about features and limitations of using CAP with H2](../java/cqn-services/persistence-services#h2){.learn-more}

::: warning Not supported for CAP Node.js. :::

## Setup & Configuration {.java} ### Using the Maven Archetype {.java} When a new CAP Java project is created with the [Maven Archetype](../java/developing-applications/building#the-maven-archetype) or with `cds init`, H2 is automatically configured as in-memory database used for development and testing in the `default` profile. ### Manual Configuration {.java} To use H2, just add a Maven dependency to the H2 JDBC driver: ```xml com.h2database h2 runtime ``` Next, configure the build to [create an initial _schema.sql_ file](../java/cqn-services/persistence-services#initial-database-schema-1) for H2 using `cds deploy --to h2 --dry`. In Spring, H2 is automatically initialized as in-memory database when the driver is present on the classpath. [Learn more about the configuration of H2 ](../java/cqn-services/persistence-services#h2){.learn-more} ## Features {.java} CAP supports most of the major features on H2: * [Path Expressions](../java/working-with-cql/query-api#path-expressions) & Filters * [Expands](../java/working-with-cql/query-api#projections) * [Localized Queries](../guides/localized-data#read-operations) * [Comparison Operators](../java/working-with-cql/query-api#comparison-operators) * [Predicate Functions](../java/working-with-cql/query-api#predicate-functions) [Learn about features and limitations of H2](../java/cqn-services/persistence-services#h2){.learn-more} # Using PostgreSQL

This guide focuses on the new PostgreSQL Service provided through *[@cap-js/postgres](https://www.npmjs.com/package/@cap-js/postgres)*, which is based on the same new database services architecture as the new [SQLite Service](databases-sqlite). This architecture brings significantly enhanced feature sets and feature parity, as documented in the [*Features* section of the SQLite guide](databases-sqlite#features). *Learn about migrating from the former `cds-pg` in the [Migration](#migration) chapter.*{.learn-more}

CAP Java 3 is tested on [PostgreSQL](https://www.postgresql.org/) 16 and most CAP features are supported on PostgreSQL. [Learn more about features and limitations of using CAP with PostgreSQL](../java/cqn-services/persistence-services#postgresql){.learn-more}

## Setup & Configuration

Run this to use [PostgreSQL](https://www.postgresql.org/) for production:

To run CAP Java on PostgreSQL, add a Maven dependency to the PostgreSQL feature in `srv/pom.xml`: ```xml com.sap.cds cds-feature-postgresql runtime ``` In order to use the CDS tooling with PostgreSQL, you also need to install the module `@cap-js/postgres`:

```sh npm add @cap-js/postgres ```

After that, you can use the `cds deploy` command to [deploy](#using-cds-deploy) to a PostgreSQL database or to [create a DDL script](#deployment-using-liquibase) for PostgreSQL.

### Auto-Wired Configuration {.node} The `@cap-js/postgres` package uses `cds-plugin` technique to auto-configure your application and use a PostgreSQL database for production. You can inspect the effective configuration using `cds env`: ```sh cds env requires.db --for production ``` Output: ```js { impl: '@cap-js/postgres', dialect: 'postgres', kind: 'postgres' } ``` [See also the general information on installing database packages](databases#setup-configuration){.learn-more} ## Provisioning a DB Instance To connect to a PostgreSQL offering from the cloud provider in Production, leverage the [PostgreSQL on SAP BTP, hyperscaler option](https://discovery-center.cloud.sap/serviceCatalog/postgresql-hyperscaler-option). For local development and testing convenience, you can run PostgreSQL in a [docker container](#using-docker).

To consume a PostgreSQL instance from a CAP Java application running on SAP BTP, consider the following: - Only the Java buildpack `java_buildpack` provided by the Cloud Foundry community allows to consume a PostgreSQL service from a CAP Java application. - By default, the `java_buildpack` initializes a PostgreSQL datasource with the Java CFEnv library. However, to work properly with CAP, the PostgreSQL datasource must be created by the CAP Java runtime and not by the buildpack. You need to disable the [datasource initialization by the buildback](https://docs.cloudfoundry.org/buildpacks/java/configuring-service-connections.html) using `CFENV_SERVICE__ENABLED: false` at your CAP Java service module. The following example shows these configuration settings applied to a CAP Java service: ::: code-group ```yaml [mta.yaml] modules: - name: bookshop-pg-srv type: java path: srv parameters: buildpack: java_buildpack properties: SPRING_PROFILES_ACTIVE: cloud JBP_CONFIG_COMPONENTS: '{jres: ["JavaBuildpack::Jre::SapMachineJRE"]}' JBP_CONFIG_SAP_MACHINE_JRE: '{ jre: { version: "17.+" } }' CFENV_SERVICE_BOOKSHOP-PG-DB_ENABLED: false ``` ::: > `BOOKSHOP-PG-DB` is the real PostgreSQL service instance name in this example.

### Using Docker You can use Docker to run a PostgreSQL database locally as follows: 1. Install and run [Docker Desktop](https://www.docker.com) 2. Create the following file in your project root directory: ::: code-group ```yaml [pg.yml] services: db: image: postgres:alpine environment: { POSTGRES_PASSWORD: postgres } ports: [ '5432:5432' ] restart: always ``` ::: 3. Create and run the docker container: ```sh docker-compose -f pg.yml up -d ```

::: tip With the introduction of [Testcontainers support](https://spring.io/blog/2023/06/23/improved-testcontainers-support-in-spring-boot-3-1) in Spring Boot 3.1, you can create PostgreSQL containers on the fly for local development or testing purposes. :::

## Service Bindings You need a service binding to connect to the PostgreSQL database. In the cloud, use given techniques to bind a cloud-based instance of PostgreSQL to your application.

For local development provide the credentials using a suitable [`cds env`](../node.js/cds-env) technique, like one of the following.

### Configure Connection Data {.java} If a PostgreSQL service binding exists, the corresponding `DataSource` is auto-configured. You can also explicitly [configure the connection data](../java/cqn-services/persistence-services#postgres-connection) of your PostgreSQL database in the _application.yaml_ file. If you run the PostgreSQL database in a [docker container](#using-docker) your connection data might look like this: ::: code-group ```yaml [srv/src/main/resources/application.yaml] spring: config.activate.on-profile: postgres-docker datasource: url: jdbc:postgresql://localhost:5432/postgres username: postgres password: postgres driver-class-name: org.postgresql.Driver ``` ::: To start the application with the new profile `postgres-docker`, the `spring-boot-maven-plugin` can be used: `mvn spring-boot:run -Dspring-boot.run.profiles=postgres-docker`. [Learn more about the configuration of a PostgreSQL database](../java/cqn-services/persistence-services#postgresql-1){ .learn-more} ### Service Bindings for CDS Tooling {.java} #### Using Defaults with `[pg]` Profile {.java} `@cds-js/postgres` comes with a set of default credentials under the profile `[pg]` that matches the defaults used in the [docker setup](#using-docker). So, if you stick to these defaults you can skip to deploying your database with: ```sh cds deploy --profile pg ``` #### In Your Private `.cdsrc-private.json` {.java} If you don't use the default credentials and want to use just `cds deploy`, you need to configure the service bindings (connection data) for the CDS tooling. Add the connection data to your private `.cdsrc-private.json`: ```json { "requires": { "db": { "kind": "postgres", "credentials": { "host": "localhost", "port": 5432, "user": "postgres", "password": "postgres", "database": "postgres" } } } } ``` ### Configure Service Bindings {.node} #### Using Defaults with `[pg]` Profile The `@cds-js/postgres` comes with default credentials under profile `[pg]` that match the defaults used in the [docker setup](#using-docker). So, in case you stick to these defaults you can skip the next sections and just go ahead, deploy your database: ```sh cds deploy --profile pg ``` Run your application: ```sh cds watch --profile pg ``` Learn more about that in the [Deployment](#deployment) chapter below.{.learn-more} #### In Your private `~/.cdsrc.json` Add it to your private `~/.cdsrc.json` if you want to use these credentials on your local machine only: ::: code-group ```json [~/.cdsrc.json] { "requires": { "db": { "[pg]": { "kind": "postgres", "credentials": { "host": "localhost", "port": 5432, "user": "postgres", "password": "postgres", "database": "postgres" } } } } } ``` ::: #### In Project `.env` Files Alternatively, use a `.env` file in your project's root folder if you want to share the same credentials with your team: ::: code-group ```properties [.env] cds.requires.db.[pg].kind = postgres cds.requires.db.[pg].credentials.host = localhost cds.requires.db.[pg].credentials.port = 5432 cds.requires.db.[pg].credentials.user = postgres cds.requires.db.[pg].credentials.password = postgres cds.requires.db.[pg].credentials.database = postgres ``` ::: ::: tip Using Profiles The previous configuration examples use the [`cds.env` profile](../node.js/cds-env#profiles) `[pg]` to allow selectively testing with PostgreSQL databases from the command line as follows: ```sh cds watch --profile pg ``` The profile name can be freely chosen, of course. ::: ## Deployment ### Using `cds deploy` Deploy your database as usual with that: ```sh cds deploy ``` Or with that if you used profile `[pg]` as introduced in the setup chapter above: ```sh cds deploy --profile pg ``` ### With a Deployer App When deploying to Cloud Foundry, this can be accomplished by providing a simple deployer app. Similar to SAP HANA deployer apps, it is auto-generated for PostgreSQL-enabled projects by running ```sh cds build --production ``` ::: details What `cds build` does… 1. Compiles the model into _gen/pg/db/csn.json_. 2. Copies required `.csv` files into _gen/pg/db/data_. 3. Adds a _gen/pg/package.json_ with this content: ```json { "dependencies": { "@sap/cds": "^8", "@cap-js/postgres": "^1" }, "scripts": { "start": "cds-deploy" } } ``` > **Note the dash in `cds-deploy`**, which is required as we don't use `@cds-dk` for deployment and runtime, so the `cds` CLI executable isn't available. ::: ### Add PostgreSQL Deployment Configuration ```sh cds add postgres ``` ::: details See what this does… 1. Adds `@cap-js/postgres` dependency to your _package.json_ `dependencies`. 2. Sets up deployment descriptors such as _mta.yaml_ to use a PostgreSQL instance deployer application. 3. Wires up the PostgreSQL service to your deployer app and CAP backend. ::: ### Deploy You can package and deploy that application, for example using [MTA-based deployment](deployment/to-cf#build-mta). ## Automatic Schema Evolution { #schema-evolution } When redeploying after you changed your CDS models, like adding fields, automatic schema evolution is applied. Whenever you run `cds deploy` (or `cds-deploy`) it executes these steps: 1. Read a CSN of a former deployment from table `cds_model`. 2. Calculate the **delta** to current model. 3. Generate and run DDL statements with: - `CREATE TABLE` statements for new entities - `CREATE VIEW` statements for new views - `ALTER TABLE` statements for entities with new or changed elements - `DROP & CREATE VIEW` statements for views affected by changed entities 4. Fill in initial data from provided _.csv_ files using `UPSERT` commands. 5. Store a CSN representation of the current model in `cds_model`. > You can disable automatic schema evolution, if necessary, by setting cds.requires.db.schema_evolution = false. ::: danger No manual altering Manually altering the database will most likely break automatic schema evolution! ::: ### Limitations Automatic schema evolution only allows changes without potential data loss. #### Allowed{.good} - Adding entities and elements - Increasing the length of Strings - Increasing the size of Integers #### Disallowed{.bad} - Removing entities or elements - Changes to primary keys - All other type changes For example the following type changes are allowed: ```cds entity Foo { anInteger : Int64; // from former: Int32 aString : String(22); // from former: String(11) } ``` ::: tip If you need to apply such disallowed changes during development, just drop and re-create your database, for example by killing it in docker and re-create it using the `docker-compose` command, [see Using Docker](#using-docker). ::: ### Dry-Run Offline You can use `cds deploy` with option `--dry` to simulate and inspect how things work. 1. Capture your current model in a CSN file: ```sh cds deploy --dry --model-only --out cds-model.csn ``` 2. Change your models, for example in *[cap/samples/bookshop/db/schema.cds](https://github.com/SAP-samples/cloud-cap-samples/blob/main/bookshop/db/schema.cds)*: ```cds entity Books { ... title : localized String(222); //> increase length from 111 to 222 foo : Association to Foo; //> add a new relationship bar : String; //> add a new element } entity Foo { key ID: UUID } //> add a new entity ``` 3. Generate delta DDL statements: ```sh cds deploy --dry --delta-from cds-model.csn --out delta.sql ``` 4. Inspect the generated SQL statements, which should look like this: ::: code-group ```sql [delta.sql] -- Drop Affected Views DROP VIEW localized_CatalogService_ListOfBooks; DROP VIEW localized_CatalogService_Books; DROP VIEW localized_AdminService_Books; DROP VIEW CatalogService_ListOfBooks; DROP VIEW localized_sap_capire_bookshop_Books; DROP VIEW CatalogService_Books_texts; DROP VIEW AdminService_Books_texts; DROP VIEW CatalogService_Books; DROP VIEW AdminService_Books; -- Alter Tables for New or Altered Columns ALTER TABLE sap_capire_bookshop_Books ALTER title TYPE VARCHAR(222); ALTER TABLE sap_capire_bookshop_Books_texts ALTER title TYPE VARCHAR(222); ALTER TABLE sap_capire_bookshop_Books ADD foo_ID VARCHAR(36); ALTER TABLE sap_capire_bookshop_Books ADD bar VARCHAR(255); -- Create New Tables CREATE TABLE sap_capire_bookshop_Foo ( ID VARCHAR(36) NOT NULL, PRIMARY KEY(ID) ); -- Re-Create Affected Views CREATE VIEW AdminService_Books AS SELECT ... FROM sap_capire_bookshop_Books AS Books_0; CREATE VIEW CatalogService_Books AS SELECT ... FROM sap_capire_bookshop_Books AS Books_0 LEFT JOIN sap_capire_bookshop_Authors AS author_1 O ... ; CREATE VIEW AdminService_Books_texts AS SELECT ... FROM sap_capire_bookshop_Books_texts AS texts_0; CREATE VIEW CatalogService_Books_texts AS SELECT ... FROM sap_capire_bookshop_Books_texts AS texts_0; CREATE VIEW localized_sap_capire_bookshop_Books AS SELECT ... FROM sap_capire_bookshop_Books AS L_0 LEFT JOIN sap_capire_bookshop_Books_texts AS localized_1 ON localized_1.ID = L_0.ID AND localized_1.locale = session_context( '$user.locale' ); CREATE VIEW CatalogService_ListOfBooks AS SELECT ... FROM CatalogService_Books AS Books_0; CREATE VIEW localized_AdminService_Books AS SELECT ... FROM localized_sap_capire_bookshop_Books AS Books_0; CREATE VIEW localized_CatalogService_Books AS SELECT ... FROM localized_sap_capire_bookshop_Books AS Books_0 LEFT JOIN localized_sap_capire_bookshop_Authors AS author_1 O ... ; CREATE VIEW localized_CatalogService_ListOfBooks AS SELECT ... FROM localized_CatalogService_Books AS Books_0; ``` ::: > **Note:** If you use SQLite, ALTER TYPE commands are not necessary and so, are not supported, as SQLite is essentially typeless. ### Generate Scripts You can use `cds deploy` with option `--script` to generate a script as a starting point for a manual migration. The effect of `--script` essentially is the same as for `--dry`, but it also allows changes that could lead to data loss and therefore are not supported in the automatic schema migration (see [Limitations](#limitations)). For generating such a script, perform the same steps as in section [Dry-Run Offline](#dry-run-offline) above, but replace the command in step 3 by ```sh cds deploy --script --delta-from cds-model.csn --out delta_script.sql ``` If your model change includes changes that could lead to data loss, there will be a warning and a respective comment is added to the dangerous statements in the resulting script. For example, deleting an element or reducing the length of an element would look like this: ::: code-group ```sql [delta_script.sql] ... -- [WARNING] this statement is lossy ALTER TABLE sap_capire_bookshop_Books DROP price; -- [WARNING] this statement could be lossy: length reduction of element "title" ALTER TABLE sap_capire_bookshop_Books ALTER title TYPE VARCHAR(11); ... ``` ::: :::warning Always check and, if necessary, adapt the generated script before you apply it to your database! ::: ## Deployment Using Liquibase { .java } You can also use [Liquibase](https://www.liquibase.org/) to control when, where, and how database changes are deployed. Liquibase lets you define database changes [in an SQL file](https://docs.liquibase.com/change-types/sql-file.html), use `cds deploy` to quickly generate DDL scripts which can be used by Liquibase. Add a Maven dependency to Liquibase in `srv/pom.xml`: ```xml org.liquibase liquibase-core runtime ``` Once `liquibase-core` is on the classpath, [Spring runs database migrations](https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto.data-initialization.migration-tool.liquibase) automatically on application startup and before your tests run. ### ① Initial Schema Version Once you're ready to release an initial version of your database schema, you can create a DDL file that defines the initial database schema. Create a `db/changelog` subfolder under `srv/src/main/resources`, place the Liquibase _change log_ file as well as the DDL scripts for the schema versions here. The change log is defined by the [db/changelog/db.changelog-master.yml](https://docs.liquibase.com/concepts/changelogs/home.html) file: ```yml databaseChangeLog: - changeSet: id: 1 author: me changes: - sqlFile: dbms: postgresql path: db/changelog/v1/model.sql ``` Use `cds deploy` to create the _v1/model.sql_ file: ```sh cds deploy --profile pg --dry --out srv/src/main/resources/db/changelog/v1/model.sql ``` Finally, store the CSN file, which corresponds to this schema version: ```sh cds deploy --model-only --dry --out srv/src/main/resources/db/changelog/v1/model.csn ``` The CSN file is needed as an input to compute the delta DDL script for the next change set. If you start your application with `mvn spring-boot:run` Liquibase initializes the database schema to version `v1`, unless it has already been initialized. ::: warning Don't change the _model.sql_ after it has been deployed by Liquibase as the [checksum](https://docs.liquibase.com/concepts/changelogs/changeset-checksums.html) of the file is validated. These files should be checked into your version control system. Follow step ② to make changes. ::: ### ② Schema Evolution { #schema-evolution-with-liquibase } If changes of the CDS model require changes on the database, you can create a new change set that captures the necessary changes. Use `cds deploy` to compute the delta DDL script based on the previous model versions (_v1/model.csn_) and the current model. Write the diff into a _v2/delta.sql_ file: ```sh cds deploy --profile pg --dry --delta-from srv/src/main/resources/db/changelog/v1/model.csn --out \ srv/src/main/resources/db/changelog/v2/model.sql ``` Next, add a corresponding change set in the _changelog/db.changelog-master.yml_ file: ```yml databaseChangeLog: - changeSet: id: 1 author: me changes: - sqlFile: dbms: postgresql path: db/changelog/v1/model.sql - changeSet: id: 2 author: me changes: - sqlFile: dbms: postgresql path: db/changelog/v2/model.sql ``` Finally, store the CSN file, which corresponds to this schema version: ```sh cds deploy --model-only --dry --out srv/src/main/resources/db/changelog/v2/model.csn ``` If you now start the application, Liquibase executes all change sets, which haven't yet been deployed to the database. For further schema versions, repeat step ②. ::: info Only compatible changes A delta DDL script is only produced for changes without potential data loss. If the changes in the model could lead to data loss, an error is raised. ::: ## Migration { .node } Thanks to CAP's database-agnostic cds.ql API, we're confident that the new PostgreSQL service comes without breaking changes. Nevertheless, please check the instructions in the [SQLite Migration guide](databases-sqlite#migration), with by and large applies also to the new PostgreSQL service. ### `cds deploy --model-only` Not a breaking change, but definitely required to migrate former `cds-pg` databases, is to prepare it for schema evolution. To do so run `cds deploy` once with the `--model-only` flag: ```sh cds deploy --model-only ``` This will...: - Create the `cds_model` table in your database. - Fill it with the current model obtained through `cds compile '*'`. ::: warning IMPORTANT: Your `.cds` models are expected to reflect the deployed state of your database. ::: ### With Deployer App When you have a SaaS application, upgrade all your tenants using the [deployer app](#with-deployer-app) with CLI option `--model-only` added to the start script command of your *package.json*. After having done that, don't forget to remove the `--model-only` option from the start script, to activate actual schema evolution. ## MTX Support ::: warning [Multitenancy](../guides/multitenancy/) and [extensibility](../guides/extensibility/) aren't yet supported on PostgreSQL. ::: # Using SAP HANA Cloud for Production [SAP HANA Cloud](https://www.sap.com/products/technology-platform/hana.html) is supported as the CAP standard database and recommended for productive use with full support for schema evolution and multitenancy. ::: warning CAP isn't validated with other variants of SAP HANA, like "SAP HANA Database as a Service" or "SAP HANA (on premise)". ::: ## Setup & Configuration

Run this to use SAP HANA Cloud for production: ```sh npm add @cap-js/hana ``` ::: details Using other SAP HANA drivers... Package `@cap-js/hana` uses the [`hdb`](https://www.npmjs.com/package/hdb) driver by default. You can override that by running [`npm add @sap/hana-client`](https://www.npmjs.com/package/@sap/hana-client), thereby adding it to your package dependencies, which then takes precedence over the default driver. ::: ::: tip Prefer `cds add` ... as documented in the [deployment guide](deployment/to-cf#_1-using-sap-hana-database), which also does the equivalent of `npm add @cap-js/hana` but in addition cares for updating `mta.yaml` and other deployment resources. :::

To use SAP HANA Cloud, [configure a module](../java/developing-applications/building#standard-modules), which includes the feature `cds-feature-hana`. For example, add a Maven runtime dependency to the `cds-feature-hana` feature: ```xml com.sap.cds cds-feature-hana runtime ``` ::: tip The [modules](../java/developing-applications/building#standard-modules) `cds-starter-cloudfoundry` and `cds-starter-k8s` include `cds-feature-hana`. ::: The datasource for HANA is then auto-configured based on available service bindings of type *service-manager* and *hana*. [Learn more about the configuration of an SAP HANA Cloud Database](../java/cqn-services/persistence-services#sap-hana){ .learn-more}

## Running `cds build` Deployment to SAP HANA is done via the [SAP HANA Deployment Infrastructure (HDI)](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-developer-guide-for-cloud-foundry-multitarget-applications-sap-business-app-studio/sap-hdi-deployer?) which in turn requires running `cds build` to generate all the deployable HDI artifacts. For example, run this in [cap/samples/bookshop](https://github.com/SAP-samples/cloud-cap-samples/tree/main/bookshop): ```sh cds build --for hana ``` Which should display this log output: ```log [cds] - done > wrote output to: gen/db/init.js gen/db/package.json gen/db/src/gen/.hdiconfig gen/db/src/gen/.hdinamespace gen/db/src/gen/AdminService.Authors.hdbview gen/db/src/gen/AdminService.Books.hdbview gen/db/src/gen/AdminService.Books_texts.hdbview gen/db/src/gen/AdminService.Currencies.hdbview gen/db/src/gen/AdminService.Currencies_texts.hdbview gen/db/src/gen/AdminService.Genres.hdbview gen/db/src/gen/AdminService.Genres_texts.hdbview gen/db/src/gen/CatalogService.Books.hdbview gen/db/src/gen/CatalogService.Books_texts.hdbview gen/db/src/gen/CatalogService.Currencies.hdbview gen/db/src/gen/CatalogService.Currencies_texts.hdbview gen/db/src/gen/CatalogService.Genres.hdbview gen/db/src/gen/CatalogService.Genres_texts.hdbview gen/db/src/gen/CatalogService.ListOfBooks.hdbview gen/db/src/gen/data/sap.capire.bookshop-Authors.csv gen/db/src/gen/data/sap.capire.bookshop-Authors.hdbtabledata gen/db/src/gen/data/sap.capire.bookshop-Books.csv gen/db/src/gen/data/sap.capire.bookshop-Books.hdbtabledata gen/db/src/gen/data/sap.capire.bookshop-Books.texts.csv gen/db/src/gen/data/sap.capire.bookshop-Books.texts.hdbtabledata gen/db/src/gen/data/sap.capire.bookshop-Genres.csv gen/db/src/gen/data/sap.capire.bookshop-Genres.hdbtabledata gen/db/src/gen/localized.AdminService.Authors.hdbview gen/db/src/gen/localized.AdminService.Books.hdbview gen/db/src/gen/localized.AdminService.Currencies.hdbview gen/db/src/gen/localized.AdminService.Genres.hdbview gen/db/src/gen/localized.CatalogService.Books.hdbview gen/db/src/gen/localized.CatalogService.Currencies.hdbview gen/db/src/gen/localized.CatalogService.Genres.hdbview gen/db/src/gen/localized.CatalogService.ListOfBooks.hdbview gen/db/src/gen/localized.sap.capire.bookshop.Authors.hdbview gen/db/src/gen/localized.sap.capire.bookshop.Books.hdbview gen/db/src/gen/localized.sap.capire.bookshop.Genres.hdbview gen/db/src/gen/localized.sap.common.Currencies.hdbview gen/db/src/gen/sap.capire.bookshop.Authors.hdbtable gen/db/src/gen/sap.capire.bookshop.Books.hdbtable gen/db/src/gen/sap.capire.bookshop.Books_author.hdbconstraint gen/db/src/gen/sap.capire.bookshop.Books_currency.hdbconstraint gen/db/src/gen/sap.capire.bookshop.Books_foo.hdbconstraint gen/db/src/gen/sap.capire.bookshop.Books_genre.hdbconstraint gen/db/src/gen/sap.capire.bookshop.Books_texts.hdbtable gen/db/src/gen/sap.capire.bookshop.Genres.hdbtable gen/db/src/gen/sap.capire.bookshop.Genres_parent.hdbconstraint gen/db/src/gen/sap.capire.bookshop.Genres_texts.hdbtable gen/db/src/gen/sap.common.Currencies.hdbtable gen/db/src/gen/sap.common.Currencies_texts.hdbtable ``` ### Generated HDI Artifacts As we see from the log output `cds build` generates these [deployment artifacts as expected by HDI](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-deployment-infrastructure-hdi-reference/sap-hdi-artifact-types-and-build-plug-ins-reference?), based on CDS models and .csv files provided in your projects: - `.hdbtable` files for entities - `.hdbview` files for views / projections - `.hdbconstraint` files for database constraints - `.hdbtabledata` files for CSV content - a few technical files required by HDI, such as [`.hdinamespace`](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-developer-guide-for-cloud-foundry-multitarget-applications-sap-business-app-studio/sap-hdi-name-space-configuration-syntax?version=2024_1_QRC&q=hdinamespace) and [`.hdiconfig`](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-developer-guide-for-cloud-foundry-multitarget-applications-sap-business-app-studio/sap-hdi-container-configuration-file?) ### Custom HDI Artifacts In addition to the generated HDI artifacts, you can add custom ones by adding according files to folder `db/src`. For example, let's add an index for Books titles... 1. Add a file `db/src/sap.capire.bookshop.Books.hdbindex` and fill it with this content: ::: code-group ```sql [db/src/sap.capire.bookshop.Books.hdbindex] INDEX sap_capire_bookshop_Books_title_index ON sap_capire_bookshop_Books (title) ``` ::: 2. Run cds build again → this time you should see this additional line in the log output: ```log [cds] - done > wrote output to: [...] gen/db/src/sap.capire.bookshop.Books.hdbindex // [!code focus] ``` [Learn more about HDI Design-Time Resources and Build Plug-ins](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-developer-guide-for-cloud-foundry-multitarget-applications-sap-business-app-studio/hdi-design-time-resources-and-build-plug-ins?){.learn-more} ## Deploying to SAP HANA There are two ways to include SAP HANA in your setup: Use SAP HANA in a [hybrid mode](#cds-deploy-hana), meaning running your services locally and connecting to your database in the cloud, or running your [whole application](deployment/) on SAP Business Technology Platform. This is possible either in trial accounts or in productive accounts. To make the following configuration steps work, we assume that you've provisioned, set up, and started, for example, your SAP HANA Cloud instance in the [trial environment](https://cockpit.hanatrial.ondemand.com). If you need to prepare your SAP HANA first, see [How to Get an SAP HANA Cloud Instance for SAP Business Technology Platform, Cloud Foundry environment](../get-started/troubleshooting#get-hana) to learn about your options. ### Prepare for Production { #configure-hana .node } To prepare the project, execute: ```sh cds add hana --for hybrid ``` This configures deployment for SAP HANA to use the _hdbtable_ and _hdbview_ formats. The configuration is added to a `[hybrid]` profile in your _package.json_. ::: tip The profile `hybrid` relates to [the hybrid testing](../advanced/hybrid-testing) scenario If you want to prepare your project for production and use the profile `production`, read the [Deploy to Cloud Foundry](deployment/) guide. ::: No further configuration is necessary for Node.js. For Java, see the [Use SAP HANA as the Database for a CAP Java Application](https://developers.sap.com/tutorials/cp-cap-java-hana-db.html#880cf07a-1788-4fda-b6dd-b5a6e5259625) tutorial for the rest of the configuration. ### Using `cds deploy` for Ad-Hoc Deployments { #cds-deploy-hana .node } `cds deploy` lets you deploy _just the database parts_ of the project to an SAP HANA instance. The server application (the Node.js or Java part) still runs locally and connects to the remote database instance, allowing for fast development roundtrips. Make sure that you're [logged in to Cloud Foundry](deployment/to-cf#deploy) with the correct target, that is, org and space. Then in the project root folder, just execute: ```sh cds deploy --to hana ``` > To connect to your SAP HANA Cloud instance use `cds watch --profile hybrid`. Behind the scenes, `cds deploy` does the following: * Compiles the CDS model to SAP HANA files (usually in _gen/db_, or _db/src/gen_) * Generates _[.hdbtabledata](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-deployment-infrastructure-hdi-reference/table-data-hdbtabledata?)_ files for the [CSV files](databases#providing-initial-data) in the project. If a _[.hdbtabledata](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-deployment-infrastructure-hdi-reference/table-data-hdbtabledata?)_ file is already present next to the CSV files, no new file is generated. * Creates a Cloud Foundry service of type `hdi-shared`, which creates an HDI container. Also, you can explicitly specify the name like so: `cds deploy --to hana:`. * Starts `@sap/hdi-deploy` locally. If you need a tunnel to access the database, you can specify its address with `--tunnel-address `. * Stores the binding information with profile `hybrid` in the _.cdsrc-private.json_ file of your project. You can use a different profile with parameter `--for`. With this information, `cds watch`/`run` can fetch the SAP HANA credentials at runtime, so that the server can connect to it. Specify `--profile` when running `cds deploy` as follows: ```sh cds deploy --to hana --profile hybrid ``` Based on these profile settings, `cds deploy` executes `cds build` and also resolves additionally binding information. If a corresponding binding exists, its service name and service key are used. The development profile is used by default. [Learn more about the deployment using HDI.](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-developer-guide-for-cloud-foundry-multitarget-applications-sap-business-app-studio/sap-hdi-deployer?){.learn-more} [Learn more about hybrid testing using service bindings to Cloud services.](../advanced/hybrid-testing#run-with-service-bindings){.learn-more} If you run into issues, see the [Troubleshooting](../get-started/troubleshooting#hana) guide. #### Deploy Parameters When using the option `--to hana`, you can specify the service name and logon information in several ways.
`cds deploy --to hana` In this case the service name and service key either come from the environment variable `VCAP_SERVICES` or are defaulted from the project name, for example, `myproject-db` with `myproject-db-key`. Service instances and key either exist and will be used, or otherwise they're created. ##### `cds deploy --to hana:myservice` This overwrites any information coming from environment variables. The service name `myservice` is used and the current Cloud Foundry client logon information is taken to connect to the system. ##### `cds deploy --vcap-file someEnvFile.json` This takes the logon information and the service name from the `someEnvFile.json` file and overwrite any environment variable that is already set. ##### `cds deploy --to hana:myservice --vcap-file someEnvFile.json` This is equivalent to `cds deploy --to hana:myservice` and ignores information coming from `--vcap-file`. A warning is printed after deploying. ### Using `cf deploy` or `cf push` { .node } See the [Deploying to Cloud Foundry](deployment/) guide for information about how to deploy the complete application to SAP Business Technology Platform, including a dedicated deployer application for the SAP HANA database. ## Native SAP HANA Features The HANA Service provides dedicated support for native SAP HANA features as follows. ### Vector Embeddings { #vector-embeddings } Vector embeddings are numerical representations that capture important features and semantics of unstructured data - such as text, images, or audio. This representation makes vector embeddings of similar data have high similarity and low distance to each other. These properties of vector embeddings facilitate tasks like similarity search, anomaly detection, recommendations and Retrieval Augmented Generation (RAG). Vector embeddings from a vector datastore such as the [SAP HANA Cloud Vector Engine](https://community.sap.com/t5/technology-blogs-by-sap/sap-hana-cloud-s-vector-engine-announcement/ba-p/13577010) can help get better generative AI (GenAI) results. This is achieved when the embeddings are used as context to the large language models (LLMs) prompts. Typically vector embeddings are computed using an **embedding model**. The embedding model is specifically designed to capture important features and semantics of a specific type of data, it also determines the dimensionality of the vector embedding space. Unified consumption of embedding models and LLMs across different vendors and open source models is provided via the [SAP Generative AI Hub](https://community.sap.com/t5/technology-blogs-by-sap/how-sap-s-generative-ai-hub-facilitates-embedded-trustworthy-and-reliable/ba-p/13596153). In CAP, vector embeddings are stored in elements of type [cds.Vector](../cds/types): ```cds entity Books : cuid { // [!code focus] title : String(111); description : LargeString; // [!code focus] embedding : Vector(1536); // vector space w/ 1536 dimensions // [!code focus] } // [!code focus] ``` At runtime, you can compute the similarity and distance of vectors in the SAP HANA vector store using the `cosineSimilarity` and `l2Distance` (Euclidean distance) functions in queries: ::: code-group ```js [Node.js] let embedding; // vector embedding as string '[0.3,0.7,0.1,...]'; let similarBooks = await SELECT.from('Books') .where`cosine_similarity(embedding, to_real_vector(${embedding})) > 0.9` ``` ```java [Java] // Vector embedding of text, for example, from SAP GenAI Hub or via LangChain4j float[] embedding = embeddingModel.embed(bookDescription).content().vector(); Result similarBooks = service.run(Select.from(BOOKS).where(b -> CQL.cosineSimilarity(b.embedding(), CQL.vector(embedding)).gt(0.9))); ``` ::: [Learn more about Vector Embeddings in CAP Java](../java/cds-data#vector-embeddings) {.learn-more} ### Geospatial Functions CDS supports the special syntax for SAP HANA geospatial functions: ```cds entity Geo as select from Foo { geoColumn.ST_Area() as area : Decimal, new ST_Point(2.25, 3.41).ST_X() as x : Decimal }; ``` *Learn more in the [SAP HANA Spatial Reference](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-spatial-reference/accessing-and-manipulating-spatial-data?).*{.learn-more} ### Spatial Grid Generators SAP HANA Spatial has some built-in [grid generator table functions](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-spatial-reference/grid-generators?). To use them in a CDS model, first define corresponding facade entities in CDS. Example for function `ST_SquareGrid`: ```cds @cds.persistence.exists entity ST_SquareGrid(size: Double, geometry: hana.ST_GEOMETRY) { geom: hana.ST_GEOMETRY; i: Integer; j: Integer; } ``` Then the function can be called, parameters have to be passed by name: ```cds entity V as select from ST_SquareGrid(size: 1.0, geometry: ST_GeomFromWkt('Point(1.5 -2.5)')) { geom, i, j }; ``` ### Functions Without Arguments SAP HANA allows to omit the parentheses for functions that don't expect arguments. For example: ```cds entity Foo { key ID : UUID; } entity Bar as select from Foo { ID, current_timestamp }; ``` Some of which are well-known standard functions like `current_timestamp` in the previous example, which can be written without parentheses in CDS models. However, there are many unknown ones, that aren't known to the compiler, for example: - `current_connection` - `current_schema` - `current_transaction_isolation_level` - `current_utcdate` - `current_utctime` - `current_utctimestamp` - `sysuuid` To use these in CDS models, you have to add the parentheses so that CDS generic support for using native features can kick in: ```cds entity Foo { key ID : UUID; } entity Bar as select from Foo { ID, current_timestamp, sysuuid() as sysid // [!code focus] }; ``` ### Regex Functions CDS supports SAP HANA Regex functions (`locate_regexpr`, `occurrences_regexpr`, `replace_regexpr`, and `substring_regexpr`), and SAP HANA aggregate functions with an additional `order by` clause in the argument list. Example: ```sql locate_regexpr(pattern in name from 5) first_value(name order by price desc) ``` Restriction: `COLLATE` isn't supported. For other functions, where the syntax isn't supported by the compiler (for example, `xmltable(...)`), a native _.hdbview_ can be used. See [Using Native SAP HANA Artifacts](../advanced/hana) for more details. ## HDI Schema Evolution CAP supports database schema updates by detecting changes to the CDS model when executing the CDS build. If the underlying database offers built-in schema migration techniques, compatible changes can be applied to the database without any data loss or the need for additional migration logic. Incompatible changes like deletions are also detected, but require manual resolution, as they would lead to data loss. | Change | Detected Automatically | Applied Automatically | | ---------------------------------- | :--------------------: | :-------------------: | | Adding fields | **Yes** | **Yes** | | Deleting fields | **Yes** | No | | Renaming fields | n/a ¹ | No | | Changing datatype of fields | **Yes** | No | | Changing type parameters | **Yes** | **Yes** | | Changing associations/compositions | **Yes** | No ² | | Renaming associations/compositions | n/a ¹ | No | | Renaming entities | n/a | No | > ¹ Rename field or association operations aren't detected as such. Instead, corresponding ADD and DROP statements are rendered requiring manual resolution activities. > > ² Changing targets may lead to renamed foreign keys. Possibly hard to detect data integrity issues due to non-matching foreign key values if target key names remain the same (for example "ID"). ::: warning No support for incompatible schema changes Currently there's no framework support for incompatible schema changes that require scripted data migration steps (like changing field constraints NULL > NOT NULL). However, the CDS build does detect those changes and renders them as non-executable statements, requesting the user to take manual resolution steps. We recommend avoiding those changes in productive environments. ::: ### Schema Evolution and Multitenancy/Extensibility There's full support for schema evolution when the _cds-mtxs_ library is used for multitenancy handling. It ensures that all schema changes during base-model upgrades are rolled out to the tenant databases. ::: warning Tenant-specific extensibility using the _cds-mtxs_ library isn't supported yet Right now, you can't activate extensions on entities annotated with `@cds.persistence.journal`. ::: ### Schema Updates with SAP HANA {#schema-updates-with-sap-hana} All schema updates in SAP HANA are applied using SAP HANA Deployment Infrastructure (HDI) design-time artifacts, which are auto-generated during CDS build execution. Schema updates using _.hdbtable_ deployments are a challenge for tables with large data volume. Schema changes with _.hdbtable_ are applied using temporary table generation to preserve the data. As this could lead to long deployment times, the support for _.hdbmigrationtable_ artifact generation has been added. The [Migration Table artifact type](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-deployment-infrastructure-hdi-reference/migration-tables-hdbmigrationtable?version=2024_1_QRC) uses explicit versioning and migration tasks. Modifications of the database table are explicitly specified in the design-time file and carried out on the database table exactly as specified. This saves the cost of an internal table-copy operation. When a new version of an already existing table is deployed, HDI performs the migration steps that haven't been applied. #### Deploy Artifact Transitions as Supported by HDI {#deploy-artifact-transitions} | Current format | hdbcds | hdbtable | hdbmigrationtable | |-------------------|:------:|:--------:|:-----------------:| | hdbcds | | yes | n/a | | hdbtable | n/a | | yes | | hdbmigrationtable | n/a | Yes | | ::: warning Direct migration from _.hdbcds_ to _.hdbmigrationtable_ isn't supported by HDI. A deployment using _.hdbtable_ is required up front. [Learn more in the **Enhance Project Configuration for SAP HANA Cloud** section.](#configure-hana){.learn-more} During the transition from _.hdbtable_ to _.hdbmigrationtable_ you have to deploy version=1 of the _.hdbmigrationtable_ artifact, which must not include any migration steps. ::: HDI supports the _hdbcds → hdbtable → hdbmigrationtable_ migration flow without data loss. Even going back from _.hdbmigrationtable_ to _.hdbtable_ is possible. Keep in mind that you lose the migration history in this case. For all transitions you want to execute in HDI, you need to specify an undeploy allowlist as described in [HDI Delta Deployment and Undeploy Allow List](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-developer-guide-for-cloud-foundry-multitarget-applications-sap-business-app-studio/hdi-delta-deployment-and-undeploy-allow-list?) in the SAP HANA documentation. :::tip Moving From _.hdbcds_ To _.hdbtable_ There a migration guide providing you step-by-step instructions for making the switch. [Learn more about Moving From _.hdbcds_ To _.hdbtable_](../cds/compiler/hdbcds-to-hdbtable){.learn-more} ::: #### Enabling hdbmigrationtable Generation for Selected Entities During CDS Build {#enabling-hdbmigrationtable-generation} If you're migrating your already deployed scenario to _.hdbmigrationtable_ deployment, you've to consider the remarks in [Deploy Artifact Transitions as Supported by HDI](#deploy-artifact-transitions). By default, all entities are still compiled to _.hdbtable_ and you only selectively choose the entities for which you want to build _.hdbmigrationtable_ by annotating them with `@cds.persistence.journal`. Example: ```cds namespace data.model; @cds.persistence.journal entity LargeBook { key id : Integer; title : String(100); content : LargeString; } ``` CDS build generates _.hdbmigrationtable_ source files for annotated entities as well as a _last-dev/csn.json_ source file representing the CDS model state of the last build. > These source files have to be checked into the version control system. Subsequent model changes are applied automatically as respective migration versions including the required schema update statements to accomplish the new target state. There are cases where you have to resolve or refactor the generated statements, like for reducing field lengths. As they can't be executed without data loss (for example, `String(100)` -> `String(50)`), the required migration steps are only added as comments for you to process explicitly. Example: ```txt >>>> Manual resolution required - DROP statements causing data loss are disabled >>>> by default. >>>> You may either: >>>> uncomment statements to allow incompatible changes, or >>>> refactor statements, e.g. replace DROP/ADD by single RENAME statement >>>> After manual resolution delete all lines starting with >>>>> -- ALTER TABLE my_bookshop_Books DROP (title); -- ALTER TABLE my_bookshop_Books ADD (title NVARCHAR(50)); ``` Changing the type of a field causes CDS build to create a corresponding ALTER TABLE statement. [Data type conversion rules](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/data-type-conversion?) are applied by the SAP HANA database as part of the deployment step. This may cause the deployment to fail if the column contents can't be converted to the new format. Examples: 1. Changing the type of a field from String to Integer may cause tenant updates to fail if existing content can't be converted. 2. Changing the type of a field from Decimal to Integer can succeed, but decimal places are truncated. Conversion fails if the content exceeds the maximum Integer length. We recommend keeping _.hdbtable_ deployment for entities where you expect low data volume. Every _.hdbmigrationtable_ artifact becomes part of your versioned source code, creating a new migration version on every model change/build cycle. In turn, each such migration can require manual resolution. You can switch large-volume tables to _.hdbmigrationtable_ at any time, keeping in mind that the existing _.hdbtable_ design-time artifact needs to be undeployed. ::: tip Sticking to _.hdbtable_ for the actual application development phase avoids lots of initial migration versions that would need to be applied to the database schema. ::: CDS build performs rudimentary checks on generated _.hdmigrationtable_ files: - CDS build fails if inconsistencies are encountered between the generated _.hdbmigrationtable_ files and the _last-dev/csn.json_ model state. For example, the last migration version not matching the table version is such an inconsistency. - CDS build fails if manual resolution comments starting with `>>>>>` exist in one of the generated _.hdbmigrationtable_ files. This ensures that manual resolution is performed before deployment. ### Native Database Clauses {#schema-evolution-native-db-clauses} Not all clauses supported by SQL can directly be written in CDL syntax. To use native database clauses also in a CAP CDS model, you can provide arbitrary SQL snippets with the annotations [`@sql.prepend` and `@sql.append`](databases#sql-prepend-append). In this section, we're focusing on schema evolution specific details. Schema evolution requires that any changes are applied by corresponding ALTER statements. See [ALTER TABLE statement reference](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/alter-table-statement-data-definition?version=2024_1_QRC) for more information. A new migration version is generated whenever an `@sql.append` or `@sql.prepend` annotation is added, changed, or removed. ALTER statements define the individual changes that create the final database schema. This schema has to match the schema defined by the TABLE statement in the _.hdbmigrationtable_ artifact. Please note that the compiler doesn't evaluate or process these SQL snippets. Any snippet is taken as is and inserted into the TABLE statement and the corresponding ALTER statement. The deployment fails in case of syntax errors. CDS Model: ```cds @cds.persistence.journal @sql.append: 'PERSISTENT MEMORY ON' entity E { ..., @sql.append: 'FUZZY SEARCH INDEX ON' text: String(100); } ``` Result in hdbmigrationtable file: ```sql == version=2 COLUMN TABLE E ( ..., text NVARCHAR(100) FUZZY SEARCH INDEX ON ) PERSISTENT MEMORY ON == migration=2 ALTER TABLE E PERSISTENT MEMORY ON; ALTER TABLE E ALTER (text NVARCHAR(100) FUZZY SEARCH INDEX ON); ``` It's important to understand that during deployment new migration versions will be applied on the existing database schema. If the resulting schema doesn't match the schema as defined by the TABLE statement, deployment fails and any changes are rolled-back. In consequence, when removing or replacing an existing `@sql.append` annotation, the original ALTER statements need to be undone. As the required statements can't automatically be determined, manual resolution is required. The CDS build generates comments starting with `>>>>` in order to provide some guidance and enforce manual resolution. Generated file with comments: ```txt == migration=3 >>>>> Manual resolution required - insert ALTER statement(s) as described below. >>>>> After manual resolution delete all lines starting with >>>>> >>>>> Insert ALTER statement for: annotation @sql.append of artifact E has been removed (previous value: "PERSISTENT MEMORY ON") >>>>> Insert ALTER statement for: annotation @sql.append of element E:text has been removed (previous value: "FUZZY SEARCH INDEX ON") ``` Manually resolved file: ```sql == migration=3 ALTER TABLE E PERSISTENT MEMORY DEFAULT; ALTER TABLE E ALTER (text NVARCHAR(100) FUZZY SEARCH INDEX OFF); ``` Appending text to an existing annotation is possible without manual resolution. A valid ALTER statement will be generated in this case. For example, appending the `NOT NULL` column constraint to an existing `FUZZY SEARCH INDEX ON` annotation generates the following statement: ```sql ALTER TABLE E ALTER (text NVARCHAR(100) FUZZY SEARCH INDEX ON NOT NULL); ``` ::: warning You can use `@sql.append` to partition your table initially, but you can't subsequently change the partitions using schema evolution techniques as altering partitions isn't supported yet. ::: ### Advanced Options The following CDS configuration options are supported to manage _.hdbmigrationtable_ generation. ::: warning This hasn't been finalized yet. ::: ```js { "hana" : { "journal": { "enable-drop": false, "change-mode": "alter" // "drop" }, // ... } } ``` The `"enable-drop"` option determines whether incompatible model changes are rendered as is (`true`) or manual resolution is required (`false`). The default value is `false`. The `change-mode` option determines whether `ALTER TABLE ... ALTER` (`"alter"`) or `ALTER TABLE ... DROP` (`"drop"`) statements are rendered for data type related changes. To ensure that any kind of model change can be successfully deployed to the database, you can switch the `"change-mode"` to `"drop"`, keeping in mind that any existing data will be deleted for the corresponding column. See [hdbmigrationtable Generation](#enabling-hdbmigrationtable-generation) for more details. The default value is `"alter"`. ## Caveats ### CSV Data Gets Overridden HDI deploys CSV data as _.hdbtabledata_ and assumes exclusive ownership of the data. It's overridden with the next application deployment; hence: ::: tip Only use CSV files for _configuration data_ that can't be changed by application users. ::: Yet, if you need to support initial data with user changes, you can use the `include_filter` option that _[.hdbtabledata](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-deployment-infrastructure-hdi-reference/table-data-hdbtabledata?version=2024_1_QRC)_ offers. ### Undeploying Artifacts As documented in the [HDI Deployer docs](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-developer-guide-for-cloud-foundry-multitarget-applications-sap-business-app-studio/hdi-delta-deployment-and-undeploy-allow-list?), an HDI deployment by default never deletes artifacts. So, if you remove an entity or CSV files, the respective tables, and content remain in the database. By default, `cds add hana` creates an `undeploy.json` like this: ::: code-group ```json [db/undeploy.json] [ "src/gen/**/*.hdbview", "src/gen/**/*.hdbindex", "src/gen/**/*.hdbconstraint", "src/gen/**/*_drafts.hdbtable", "src/gen/**/*.hdbcalculationview" ] ``` ::: If you need to remove deployed CSV files, also add this entry: ::: code-group ```json [db/undeploy.json] [ [...] "src/gen/**/*.hdbtabledata" ] ``` ::: *See this [troubleshooting](../get-started/troubleshooting#hana-csv) entry for more information.*{.learn-more} ### SAP HANA Cloud System Limits All limitations for the SAP HANA Cloud database can be found in the [SAP Help Portal](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/system-limitations?version=2024_2_QRC). ### Native Associations For SAP HANA, CDS associations are by default reflected in the respective database tables and views by _Native HANA Associations_ (HANA SQL clause `WITH ASSOCIATIONS`). CAP no longer needs these native associations (provided you use the new database service _@cap-js/hana_ for the CAP Node.js stack). Unless you explicitly use them in other native HANA objects, we recommend switching off the generation of native HANA associations, as they increase deploy times: They need to be validated in the HDI deployment, and they can introduce indirect dependencies between other objects, which can trigger other unnecessary revalidations or even unnecessary drop/create of indexes. By switching them off, all this effort is saved. ::: code-group ```json [package.json] { "cds": { "sql": { "native_hana_associations": false } } } ``` ```json [cdsrc.json] { "sql": { "native_hana_associations": false } } ``` ::: For new projects, `cds add hana` automatically adds this configuration. ::: warning Initial full table migration Be aware, that the first deployment after this **configuration change may take longer**. For each entity with associations, the respective database object will be touched (DROP/CREATE for views, full table migration via shadow table and data copy for tables). This is also the reason why we haven't changed the default so far. Subsequent deployments will benefit, however. ::: # Localization, i18n Guides you through the steps to internationalize your application to provide localized versions with respect to both Localized Models as well as Localized Data. _'Localization'_ is a means to adapting your app to the languages of specific target markets. This guide focuses on static texts such as labels. See [CDS](../cds/) and [Localized Data](localized-data) for information about how to manage and serve actual payload data in different translations. ## Externalizing Texts Bundles All you have to do to internationalize your models is to externalize all of your literal texts to text bundles and refer to the respective keys from your models as annotation values. Here is a sample of a model and the corresponding bundle. ::: code-group ```cds [srv/my-service.1.cds] service Bookshop { entity Books @( UI.HeaderInfo: { Title.Label: '{i18n>Book}', TypeName: '{i18n>Book}', TypeNamePlural: '{i18n>Books}', }, ){/*...*/} } ``` ::: ::: code-group ```properties [_i18n/i18n.properties] Book = Book Books = Books foo = Foo ``` ::: > You can define the keys of your properties entries. [Learn more about annotations in CSN.](../cds/csn#annotations){ .learn-more} Then you can translate the texts in localized bundles, each with a language/locale code appended to its name, for example: ```sh _i18n/ i18n.properties # dev main --› 'default fallback' i18n_en.properties # English --› 'default language' i18n_de.properties # German i18n_zh_TW.properties # Traditional Chinese ... ``` ## Where to Place Text Bundles? Recommendation is to put your properties files in a folder named `_i18n` in the root of your project, as in this example: ```zsh bookshop/ ├─ _i18n/ │ ├─ i18n_en.properties │ ├─ i18n_de.properties │ ├─ i18n_fr.properties │ └─ i18n.properties │ ... ``` By default, text bundles are fetched from folders named *_i18n* or *i18n* in the neighborhood of models, i.e. all folders that contain `.cds` sources or parent folders thereof. For example, given the following project layout and sources: ```zsh bookshop/ ├─ app/ │ ├─ browse/ │ │ └─ fiori.cds │ ├─ common.cds │ └─ index.cds ├─ srv/ │ ├─ admin-service.cds │ └─ cat-service.cds ├─ db/ │ └─ schema.cds └─ readme.md ``` We will be loading i18n bundles from all of these locations, if exist: ``` bookshop/app/browse/_i18n bookshop/app/_i18n bookshop/srv/_i18n bookshop/db/_i18n bookshop/_i18n ``` [Learn more about the underlying machinery in the reference docs for `cds.i18n`](../node.js/cds-i18n){.learn-more} ## CSV-Based Text Bundles For smaller projects you can use CSV files instead of _.properties_ files, which you can easily edit in _Excel_, _Numbers_, etc. The format is as follows: | key | en | de | zh_CN | ... | | --- | --- | --- | --- | --- | | Book | Book | Buch | ... | | Books | Books | Bücher | ... | | ... | {} With this CSV source: ```csv key;en;de;zh_CN;... Book;Book;Buch;... Books;Books;Bücher;... ... ``` ## Merging Algorithm Each localized model is constructed by applying: 1. The _default fallback_ bundle (that is, *i18n.properties*), then ... 2. The _default language_ bundle (usually *i18n_en.properties*), then ... 3. The requested bundle (for example, *i18n_de.properties*) In that order. So, the complete stack of overlaid models for the given example would look like this (higher ones override lower ones): | Source | Content | |:--- |:--- | | *_i18n/i18n_de.properties* | specific language bundle | | *_i18n/i18n_en.properties* | default language bundle | | *_i18n/i18n.properties* | default fallback bundle | | *srv/my-service.cds* | service definition | | *db/schema.cds* | underlying data model | ::: tip Set default language The _default language_ is usually `en` but can be overridden by configuring cds.i18n.default_language in your project's _package.json_. ::: ## Merging Reuse Bundles If your application is [importing models from a reuse package](extensibility/composition), that package comes with its own language bundles for localization. These are applied upon import, so they can be overridden in your models as well as in your language bundles and their translations. For example, assuming that your data model imports from a _foundation_ package, then the overall stack of overlays would look like this: | Source | |:--- | | *./_i18n/i18n_de.properties* | | *./_i18n/i18n_en.properties* | | *./_i18n/i18n.properties* | | *./srv/my-service.cds* | | *./db/schema.cds* | | *foundation/_i18n/i18n_de.properties* | | *foundation/_i18n/i18n_en.properties* | | *foundation/_i18n/i18n.properties* | | *foundation/index.cds* | | *foundation/\.cds* | | *foundation/\.cds* | | ... | ## Determining User Locales { #user-locale} Upon incoming requests at runtime, the user's preferred language is determined as follows: 1. Read the preferred language from the first of: 1. The value of the `sap-locale` URL parameter, if present. 2. The value of the `sap-language` URL parameter, but only if it's `1Q`, `2Q` or `3Q` as described below. 3. The first entry from the request's `Accept-Language` header. 4. The `default_language` configured on the app level. 2. Narrow to normalized locales as described below. ::: tip CAP Node.js accepts formats following the available standards of POSIX and RFC 1766, and transforms them into normalized locales. CAP Java only accepts language codes following the standard of RFC 1766 (or [IETF's BCP 47](https://www.rfc-editor.org/rfc/bcp/bcp47.txt)). ::: ## Normalized Locales { #normalized-locales} To reduce the number of required translations, most determined locales are normalized by narrowing them to their main language codes only, for example, `en_US`, `en_CA`, `en_AU` → `en`, except for these preserved language codes: | Locale | Language | | --- | --- | | zh_CN | Chinese - China | | zh_HK | Chinese - Hong Kong, China | | zh_TW | Chinese traditional - Taiwan, China | | en_GB | English - English | | fr_CA | French - Canada | | pt_PT | Portuguese - Portugal | | es_CO | Spanish - Colombia | | es_MX | Spanish - Mexico | | en_US_x_saptrc | SAP tracing translations w/ `sap-language=1Q` | | en_US_x_sappsd | SAP pseudo translations w/ `sap-language=2Q` | | en_US_x_saprigi | Rigi language w/ `sap-language=3Q` | #### Configuring Normalized Locales For CAP Node.js, the list of preserved locales is configurable, for example in the _package.json_ file, using the configuration option cds.i18n.preserved_locales as follows: ```jsonc {"cds":{ "i18n": { "preserved_locales": [ "en_GB", "fr_CA", "pt_PT", "pt_BR", "zh_CN", "zh_HK", "zh_TW" ] } }} ``` In this example we removed `es_CO` and `es_MX` from the list, and added `pt_BR`. In CAP Java the preserved locales can be configured via the cds.locales.normalization.includeList [property](../java/developing-applications/properties#cds-locales-normalization). ::: warning *Note:* However this list is configured, ensure to have translations for the listed locales, as the fallback language will otherwise be `en`. ::: #### Use Hyphens in File Names Due to the ambiguity regarding standards, for example, the usage of hyphens (`-`) in contrast to underscores (`_`), CAP follows the approach of the [SAP Translation Hub](https://discovery-center.cloud.sap/serviceCatalog/sap-translation-hub). Using that approach, CAP normalizes locales to **underscores** as our de facto standard. In effect, this means: - We support incoming locales as [language tags](https://www.ietf.org/rfc/bcp/bcp47.txt) using hyphens to separate sub tags ¹, for example `en-GB`. - We always normalize these to underscores, which is `en_GB`. - Always use underscores in filenames, for example, `i18n_en_GB.properties` ¹ CAP Node.js also supports underscore separated tags, for example `en_GB`.

# Localized Data This guide extends the localization/i18n of static content, such as labels or messages, to serve localized versions of actual application data. Localized data refers to the maintenance of different translations of textual data and automatically fetching the translations matching the users' preferred language, with per-row fallback to default languages, if the required translations aren't available. Language codes are in ISO 639-1 format. > Find a **working sample** at . ## Declaring Localized Data Use the `localized` modifier to mark entity elements that require translated texts. ```cds entity Books { key ID : UUID; title : localized String; descr : localized String; price : Decimal; currency : Currency; } ``` [Find this source also in **cap/samples**.](https://github.com/sap-samples/cloud-cap-samples/blob/ea6e27481071a765dfd701ddb239ed89b92bf426/bookshop/db/schema.cds#L4-L7){ .learn-more} ::: warning _Restriction_ If you want to use the `localized` modifier, the entity's keys must not be associations. ::: > `localized` in entity sub elements isn't currently supported and is ignored. > This includes `localized` in structured elements and structured types. ## Behind the Scenes The `cds` compiler automatically unfolds the previous definition as follows, applying the basic mechanisms of [Managed Compositions](../cds/cdl#managed-compositions), and [Scoped Names](../cds/cdl#scoped-names): First, a separate _Books.texts_ entity is added to hold translated texts: ```cds entity Books.texts { key locale : sap.common.Locale; key ID : UUID; //= source's primary key title : String; descr : String; } ``` [See the definition of `sap.common.Locale`.](../cds/common#locale-type){ .learn-more} Second, the source entity is extended with associations to _Books.texts_: ```cds extend entity Books with { texts : Composition of many Books.texts on texts.ID=ID; localized : Association to Books.texts on localized.ID=ID and localized.locale = $user.locale; } ``` The composition `texts` points to all translated texts for the given entity, whereas the `localized` association points to the translated texts and is narrowed to the request's locale. Third, views are generated in SQL DDL to easily read localized texts with an equivalent fallback: ```cds entity localized.Books as select from Books {*, coalesce (localized.title, title) as title, coalesce (localized.descr, descr) as descr }; ``` ::: warning Note: In contrast to former versions, with CDS compiler v2 we don't add such entities to CSN anymore, but only on generated SQL DDL output. ::: ### Resolving localized texts via views As we already mentioned, the CDS compiler is already creating views that resolve the translated texts internally. Once a CDS runtime detects a request with a user locale, it uses those views instead of the table of the involved entity. Note that SQLite doesn't support locales like _SAP HANA_ does. For _SQLite_, additional views are generated for different languages. Currently those views are generated for the locales 'de' and 'fr' and the default locale is handled as 'en'. ```json "i18n": { "for_sqlite": ["en", ...] } ``` > In _package.json_ put this snippet in the `cds` block, but don't do so for _.cdsrc.json_. > For testing with SQLite: Make sure that the _Books_ table contains the English texts and that the other languages go into the _Books.texts_ table. For _H2_, you need to use the property as follows. ```json "i18n": { "for_sql": ["en", ...] } ``` ### Resolving search over localized texts at runtime { #resolving-localized-texts-at-runtime} Although the approach with the generated localized views is very convenient, it's limited on SQLite and shows suboptimal performance with large data sets on _SAP HANA_. Especially for search operations the performance penalty is very critical. Therefore, both CAP runtimes have implemented a solution targeted for search operations. If the `localized` association of your entity is present and accessible by the given CQL statement, the runtimes generate SQL statements that resolve the localized texts. This is optimized for the underlying database. When your CQL queries select entities directly there is no issue as the `localized` association is automatically accessible in an entity with localized elements. If your CQL query selects from a view, it is important that your views' projection preserves the `localized` association. The following view definitions preserve the `localized` association in the view, allowing you to optimize query execution, or for broader language support on SQLite, H2, and PostgreSQL. **Preferred -** Exclude elements that mustn't be exposed: ```cds entity OpenBookView as select from Books {*} excluding { price, currency }; ``` Include the `localized` association: ```cds entity ClosedBookView as select from Books { ID, title, descr, localized }; ``` ### Base Entities Stay Intact In contrast to similar strategies, all texts aren't externalized but the original texts are kept in the source entity. This saves one join when reading localized texts with fallback to the original ones. ### Extending *.texts* Entities { #extending-texts-entities} It's possible to collectively extend all generated *.texts* entities by extending the aspect `sap.common.TextsAspect`, which is defined in [*common.cds*](../cds/common#texts-aspects). For example, the aspect can be used to add an association to the `Languages` code list entity, or to add flags that help you to control the translation process. Example: ```cds extend sap.common.TextsAspect with { language : Association to sap.common.Languages on language.code = locale; } ``` The earlier description is simplified, *.texts* entities are generated with an include on `sap.common.TextsAspect`, if the aspect exists. For the *Books* entity, the generated *.texts* entity looks like: ```cds entity Books.texts : sap.common.TextsAspect { key ID : UUID; title : String; descr : String; } ``` When the include is expanded, the key element `locale` is inserted into *.texts* entities, alongside all the other elements that have been added to `sap.common.TextsAspect` via extensions. ```cds entity Books.texts { // from sap.common.TextsAspect key locale: sap.common.Locale; language : Association to sap.common.Languages on language.code = locale; // from Books key ID : UUID; title : String; descr : String; } ``` It isn't allowed to extend `sap.common.TextsAspect` with * [Managed Compositions of Aspects](../cds/cdl#managed-compositions) * localized elements * key elements For entities that have an annotation `@fiori.draft.enabled`, the corresponding *.texts* entities also include the aspect, but the element `locale` isn't marked as a key and an element `key ID_texts : UUID` is added. ## Pseudo var `$user.locale` { #user-locale} [`$user.locale`]: #user-locale As shown in the second step, the pseudo variable `$user.locale` is used to refer to the user's preferred locale and join matching translations from `.texts` tables. This pseudo variable allows expressing such queries in a database-independent way, which is realized in the service runtimes as follows: ### Determining `$user.locale` from Inbound Requests The user's preferred locale is determined from request parameters, user settings, or the _accept-language_ header of inbound requests [as explained in the Localization guide](i18n#user-locale). ### Programmatic Access to `$user.locale` The resulting [normalized locale](i18n#normalized-locales) is available programmatically, in your event handlers. * Node.js: `req.locale` * Java: `eventContext.getParameterInfo().getLocale()` ### Propagating `$user.locale` to Databases {#propagating-of-user-locale} [propagation]: #propagating-of-user-locale Finally, the [normalized locale](i18n#normalized-locales) is **propagated** to underlying databases using session variables, that is, `$user.locale` translates to `session_context('locale')` in native SQL of SAP HANA and most databases. Not all databases support session variables. For example, for _SQLite_ we currently would just create stand-in views for selected languages. With that, the APIs are kept stable but have restricted feature support. ## Reading Localized Data Given the asserted unfolding and user locales propagated to the database, you can read localized data as follows: ### In Agnostic Code Read _original_ texts, that is, the ones in the originally created data entry: ```sql SELECT ID, title, descr from Books ``` ### For End Users Reading texts for end users uses the `localized` association, which requires prior [propagation] of [`$user.locale`] to the underlying database. Read _localized_ texts in the user's preferred language: ```sql SELECT ID, localized.title, localized.descr from Books ``` ### For Translation UIs Translation UIs read and write texts in all languages, independent from the current user's preferred one. They use the to-many `texts` association, which is independent from [`$user.locale`]. Read texts in **different** translations: ```sql SELECT ID, texts[locale='fr'].title, texts[locale='fr'].descr from Books ``` Read texts in **all** translations: ```sql SELECT ID, texts.locale, texts.title, texts.descr from Books ``` ## Serving Localized Data The generic handlers of the service runtimes automatically serve read requests from `localized` views. Users see all texts in their preferred language or the fallback language. [See also **Enabling Draft for Localized Data**.](../advanced/fiori#draft-for-localized-data){ .learn-more} For example, given this service definition: ```cds using { Books } from './books'; service CatalogService { entity BooksList as projection on Books { ID, title, price }; entity BooksDetails as projection on Books; entity BooksShort as projection on Books { ID, price, substr(title, 0, 10) as title : localized String(10), }; } ``` ### `localized.` Helper Views For each exposed entity in a service definition, and all intermediate views, a corresponding `localized.` entity is created. It has the same query clauses and all annotations, except for the `from` clause being redirected to the underlying entity's `localized.` counterpart. A helper view is only created if the corresponding entity contains at least one element with a `localized` property, or it exposes an association to an entity that is localized. You may need to cast an element if that property is not propagated, for example for expressions such as in `CatalogService.BooksShort`. ```cds using { localized.Books } from './books_localized'; entity localized.CatalogService.BooksList as SELECT from localized.Books { ID, title, price }; entity localized.CatalogService.BooksDetails as SELECT from localized.Books; entity localized.CatalogService.BooksShort as SELECT from localized.Books { ID, price, substr(title, 0, 10) as title : localized String(10), }; ``` ::: warning Note: Note that these `localized.` entities are not part of CSN and aren't exposed through OData. They are only generated for SQL. ::: ### Read Operations The generic handlers in the service framework will automatically redirect all incoming read requests to the `localized_` helper views in the SQL database, unless in SAP Fiori draft mode. The `@cds.localized: false` annotation can be used to explicitly switch off the automatic redirection to the localized views. All incoming requests to an entity annotated with `@cds.localized: false` will directly access the base entity. ```cds using { Books } from './books'; service CatalogService { @cds.localized: false //> direct access to base entity; all fields are non-localized defaults entity BooksDetails as projection on Books; } ``` In Node.js applications, for requests with an `$expand` query option on entities annotated with `@cds.localized: false`, the expanded properties are not translated. ```http GET /BooksDetails?$expand=authors //> all fields from authors are non-localized defaults, if BooksDetails is annotated with `@cds.localized: false` ``` ### Write Operations Since the corresponding text table is linked through composition, you can use deep inserts or upserts to fill in language-specific texts. ```http POST /Entity HTTP/1.1 Content-Type: application/json { "name": "Some name", "description": "Some description", "texts": [ {"name": "Ein Name", "description": "Eine Beschreibung", "locale": "de"} ] } ``` If you want to add a language-specific text to an existing entity, perform a `POST` request to the text table of the entity through navigation. ```http POST /Entity()/texts HTTP/1.1 Content-Type: application/json { {"name": "Ein Name", "description": "Eine Beschreibung", "locale": "de"} } ``` ### Update Operations To update the language-specific texts of an entity along with the default fallback text, you can perform a deep update as a `PUT` or `PATCH` request to the entity through navigation. ```http PUT/PATCH /Entity() HTTP/1.1 Content-Type: application/json { "name": "Some new name", "description": "Some new description", "texts": [ {"name": "Ein neuer Name", "description": "Eine neue Beschreibung", "locale": "de"} ] } ``` To update a single language-specific text field, perform a `PUT` or a `PATCH` request to the entity's text field via navigation. ```http PUT/PATCH /Entity()/texts(ID=,locale='')/ HTTP/1.1 Content-Type: application/json { {"name": "Ein neuer Name"} ] } ``` ::: warning *Note:* Accepted language codes in the `locale` property need to follow the [BCP 47](https://www.rfc-editor.org/rfc/bcp/bcp47.txt) standard but use __underscore__ (`_`) instead of __hyphen__ (`-`), for example `en_GB`. ::: ### Delete Operations To delete a locale's language-specific texts of an entity, perform a `DELETE` request to the entity's texts table through navigation. Specify the entity's key and the locale that you want to delete. ```http DELETE /Entity()/texts(ID=,locale='') HTTP/1.1 ``` ## Nested Localized Data The definition of books has a `currency` element that is effectively an association to the `sap.common.Currencies` code list entity, which in turn has localized texts. Find the respective definitions in the reference docs for `@sap/cds/common`, in the section on [Common Code Lists](../cds/common#code-lists). Upon unfolding, all associations to other entities with localized texts are automatically redirected as follows: ```cds entity localized.Currencies as select from Currencies AS c {* /*...*/}; entity localized.Books as select from Books AS p mixin { // association is redirected to localized.Currencies country : Association to localized.Currencies on country = p.country; } into {* /*...*/}; ``` Given that, nested localized data can be easily read with independent fallback logic: ```sql SELECT from localized.Books { ID, title, descr, currency.name as currency } where title like '%pen%' or currency.name like '%land%' ``` In the result sets for this query, values for `title`, `descr`, as well as the `currency` name are localized. ## Adding Initial Data To add initial data, two _.csv_ files are required. The first _.csv_ file, for example _Books.csv_, should contain all the data in the default language. The second file, for example _Books_texts.csv_ (please note **_texts** in the file name) should contain the translated data in all other languages your application is using. For example, _Books.csv_ can look as follows: ::: code-group ```csv [Books.csv] ID;title;descr;author_ID;stock;price;currency_code;genre_ID 201;Wuthering Heights;Wuthering Heights, Emily Brontë's only novel ...;101;12;11.11;GBP;11 207;Jane Eyre;Jane Eyre is a novel by English writer ...;107;11;12.34;GBP;11 251;The Raven;The Raven is a narrative poem by ...;150;333;13.13;USD;16 252;Eleonora;Eleonora is a short story by ...;150;555;14;USD;16 271;Catweazle;Catweazle is a British fantasy ...;170;22;150;JPY;13 ... ``` ::: This is the corresponding _Books_texts.csv_: ::: code-group ```csv [Books_texts.csv] ID;locale;title;descr 201;de;Sturmhöhe;Sturmhöhe (Originaltitel: Wuthering Heights) ist der einzige Roman... 201;fr;Les Hauts de Hurlevent;Les Hauts de Hurlevent (titre original : Wuthering Heights)... 207;de;Jane Eyre;Jane Eyre. Eine Autobiographie (Originaltitel: Jane Eyre. An Autobiography)... 252;de;Eleonora;Eleonora ist eine Erzählung von Edgar Allan Poe. Sie wurde 1841... ... ``` ::: # Temporal Data CAP provides out-of-the-box support for declaring and serving date-effective entities with application-controlled validity, in particular to serve as-of-now and time-travel queries. Temporal data allows you to maintain information relating to past, present, and future application time. Built-in support for temporal data follows the general principle of CDS to capture intent with models while staying conceptual, concise, and comprehensive, and minimizing pollution by technical artifacts. > For an introduction to this topic, see [Temporal database](https://en.wikipedia.org/w/index.php?title=Temporal_database&oldid=911558203) (Wikipedia) and [Temporal features in SQL:2011](https://files.ifi.uzh.ch/dbtg/ndbs/HS17/SQL2011.pdf). ## Starting with 'Timeless' Models {#timeless-model} For the following explanation, let's start with a base model to manage employees and their work assignments, which is free of any traces of temporal data management. ### Timeless Model ::: code-group ```cds [timeless-model.cds] namespace com.acme.hr; using { com.acme.common.Persons } from './common'; entity Employees : Persons { jobs : Composition of many WorkAssignments on jobs.empl=$self; job1 : Association to one /*of*/ WorkAssignments; } entity WorkAssignments { key ID : UUID; role : String(111); empl : Association to Employees; dept : Association to Departments; } entity Departments { key ID : UUID; name : String(111); head : Association to Employees; members : Association to many Employees on members.jobs.dept = $self; } ``` ::: > An employee can have several work assignments at the same time. > Each work assignment links to one department. ### Timeless Data A set of sample data entries for this model, which only captures the latest state, can look like this: ![Alice has the job a a developer and consultant. Bob is a builder. Alice works in her roles for the departments core development and app development. Bob's work assignment is linked to the construction department.](assets/temporal-data/timeless-data.drawio.svg) > Italic titles indicate to-one associations; actual names of the respective foreign key columns in SQL are `job1_ID`, `empl_ID`, and `dept_ID`. ## Declaring Temporal Entities _Temporal Entities_ represent _logical_ records of information for which we track changes over time by recording each change as individual _time slices_ in the database with valid from/to boundaries. For example, we could track the changes of Alice's primary work assignment _WA1_ over time: ![Alice progressed from developer to senior developer to architect.](assets/temporal-data/time-slices.drawio.svg) ::: tip Validity periods are expected to be **non-overlapping** and **closed-open** intervals; same as in SQL:2011. ::: ### Using Annotations `@cds.valid.from/to` To track temporal data, just add a pair of date/time elements to the respective entities annotated with `@cds.valid.from/to`, as follows: ```cds entity WorkAssignments { //... start : Date @cds.valid.from; end : Date @cds.valid.to; } ``` ::: tip The annotation pair `@cds.valid.from/to` actually triggers the built-in mechanisms for [serving temporal data](#serving-temporal-data). It specifies which elements form the **application-time** period, similar to SQL:2011. ::: ### Using Common Aspect `temporal` Alternatively, use the predefined aspect [`temporal`](../cds/common#aspect-temporal) to declare temporal entities: ```cds using { temporal } from '@sap/cds/common'; entity WorkAssignments : temporal {/*...*/} ``` Aspect [`temporal`](../cds/common#aspect-temporal) is defined in _[@sap/cds/common](../cds/common)_ as follows: ```cds aspect temporal { validFrom : Timestamp @cds.valid.from; validTo : Timestamp @cds.valid.to; } ``` ### Separate Temporal Details The previous samples would turn the whole _WorkAssignment_ entity into a temporal one. Frequently though, only some parts of an entity are temporal, while others stay timeless. You can reflect this by separating temporal elements from non-temporal ones: ```cds entity WorkAssignments { // non-temporal head entity key ID : UUID; empl : Association to Employees; details : Composition of WorkDetails on details.ID = $self.ID; } entity WorkDetails : temporal { // temporal details entity key ID : UUID; // logical record ID role : String(111); dept : Association to Departments; } ``` The data situation would change as follows: ![Alice has two work assignments. Her first work assignment is stable but the roles in this assignment change over time. She progressed from developer to senior developer to architect. Each role has specific validity defined.](assets/temporal-data/temporal-details.drawio.svg) ## Serving Temporal Data We expose the entities from the following timeless model in a service as follows: ::: code-group ```cds [service.cds] using { com.acme.hr } from './temporal-model'; service HRService { entity Employees as projection on hr.Employees; entity WorkAssignments as projection on hr.WorkAssignments; entity Departments as projection on hr.Departments; } ``` ::: > You can omit composed entities like _WorkAssignments_ from the service, as they would get [auto-exposed](providing-services#auto-exposed-entities) automatically.

## Reading Temporal Data ### As-of-now Queries READ requests without specifying any temporal query parameter will automatically return data valid _as of now_. For example, assumed the following OData query to read all employees with their current work assignments is processed on March 2019: ```cds GET Employees? $expand=jobs($select=role&$expand=dept($select=name)) ``` The values of `$at`, and so also the respective session variables, would be set to, for example: | | | | |--------------|----------------------------------|----------------------------| | `$at.from` = | _session_context('valid-from')_= | _2019-03-08T22:11:00Z_ | | `$at.to` = | _session_context('valid-to')_ = | _2019-03-08T22:11:00.001Z_ | The result set would be: ```json [ { "ID": "E1", "name": "Alice", "jobs": [ { "role": "Architect", "dept": {"name": "Core Development"}}, { "role": "Consultant", "dept": {"name": "App Development"}} ]}, { "ID": "E2", "name": "Bob", "jobs": [ { "role": "Builder", "dept": {"name": "Construction"}} ]} ] ``` ### Time-Travel Queries We can run the same OData query as in the previous sample to read a snapshot data as valid on January 1, 2017 using the `sap-valid-at` query parameter: ```cds GET Employees?sap-valid-at=date'2017-01-01' $expand=jobs($select=role&$expand=dept($select=name)) ``` The values of `$at` and hence the respective session variables would be set to, for example: | | | | |--------------|----------------------------------|----------------------------| | `$at.from` = | _session_context('valid-from')_= | _2017-01-01T00:00:00Z_ | | `$at.to` = | _session_context('valid-to')_ = | _2017-01-01T00:00:00.001Z_ | The result set would be: ```json [ { "ID": "E1", "name": "Alice", "jobs": [ { "role": "Developer", "dept": {"name": "Core Development"}}, { "role": "Consultant", "dept": {"name": "App Development"}} ]}, ... ] ``` ::: warning Time-travel queries aren't supported on SQLite due to the lack of *session_context* variables. ::: ### Time-Period Queries We can run the same OData query as in the previous sample to read all history of data as valid since 2016 using the `sap-valid-from` query parameter: ```cds GET Employees?sap-valid-from=date'2016-01-01' $expand=jobs($select=role&$expand=dept($select=name)) ``` The result set would be: ```json [ { "ID": "E1", "name": "Alice", "jobs": [ { "role": "Developer", "dept": {"name": "App Development"}}, { "role": "Developer", "dept": {"name": "Core Development"}}, { "role": "Senior Developer", "dept": {"name": "Core Development"}}, { "role": "Consultant", "dept": {"name": "App Development"}} ]}, ... ] ``` > You would add `validFrom` in such time-period queries, for example: ```cds GET Employees?sap-valid-from=date'2016-01-01' $expand=jobs($select=validFrom,role,dept/name) ``` ::: warning Time-series queries aren't supported on SQLite due to the lack of *session_context* variables. ::: ::: tip Writing temporal data must be done in custom handlers. ::: ### Transitive Temporal Data The basic techniques and built-in support for reading temporal data serves all possible use cases with respect to as-of-now and time-travel queries. Special care has to be taken though if time-period queries transitively expand across two or more temporal data entities. As an example, assume that both, _WorkAssignments_ and _Departments_ are temporal: ```cds using { temporal } from '@sap/cds/common'; entity WorkAssignments : temporal {/*...*/ dept : Association to Departments; } entity Departments : temporal {/*...*/} ``` When reading employees with all history since 2016, for example: ```cds GET Employees?sap-valid-from=date'2016-01-01' $expand=jobs( $select=validFrom,role&$expand=dept( $select=validFrom,name ) ) ``` The results for `Alice` would be: ```json [ { "ID": "E1", "name": "Alice", "jobs": [ { "validFrom":"2014-01-01", "role": "Developer", "dept": [ {"validFrom":"2013-04-01", "name": "App Development"} ]}, { "validFrom":"2017-01-01", "role": "Consultant", "dept": [ {"validFrom":"2013-04-01", "name": "App Development"} ]}, { "validFrom":"2017-01-01", "role": "Developer", "dept": [ {"validFrom":"2014-01-01", "name": "Tech Platform Dev"}, {"validFrom":"2017-07-01", "name": "Core Development"} ]}, { "validFrom":"2017-04-01", "role": "Senior Developer", "dept": [ {"validFrom":"2014-01-01", "name": "Tech Platform Dev"}, {"validFrom":"2017-07-01", "name": "Core Development"} ]}, { "validFrom":"2018-09-15", "role": "Architect", "dept": [ {"validFrom":"2014-01-01", "name": "Tech Platform Dev"}, {"validFrom":"2017-07-01", "name": "Core Development"} ]} ]}, ... ] ``` That is, all-time slices for changes to departments since 2016 are repeated for each time slice of work assignments in that time frame, which is a confusing and redundant piece of information. You can fix this by adding an alternative association to departments as follows: ```cds using { temporal } from '@sap/cds/common'; entity WorkAssignments : temporal {/*...*/ dept : Association to Departments; dept1 : Association to Departments on dept1.id = dept.id and dept1.validFrom <= validFrom and validFrom < dept1.validTo; } entity Departments : temporal {/*...*/} ``` ## Primary Keys of Time Slices While timeless entities are uniquely identified by the declared primary `key` — we call that the _conceptual_ key in CDS — time slices are uniquely identified by _the conceptual `key` **+** `validFrom`_. In effect the SQL DDL statement for the _WorkAssignments_ would look like this: ```sql CREATE TABLE com_acme_hr_WorkAssignments ( ID : nvarchar(36), validFrom : timestamp, validTo : timestamp, -- ... PRIMARY KEY ( ID, validFrom ) ) ``` In contrast to that, the exposed API preserves the timeless view, to easily serve as-of-now and time-travel queries out of the box [as described above](#serving-temporal-data): ```xml ... ``` Reading an explicit time slice can look like this: ```sql SELECT from WorkAssignments WHERE ID='WA1' and validFrom='2017-01-01' ``` Similarly, referring to individual time slices by an association: ```cds entity SomeSnapshotEntity { //... workAssignment : Association to WorkAssignments { ID, validFrom } } ```

# CAP Security Guide This guide addresses - Developers and operators - User administrators - Security experts of CAP applications who need to understand how to develop, deploy and operate CAP applications in a secure way.
# CDS-based Authorization Authorization means restricting access to data by adding respective declarations to CDS models, which are then enforced in service implementations. By adding such declarations, we essentially revoke all default access and then grant individual privileges. ## Authentication as Prerequisite { #prerequisite-authentication} In essence, authentication verifies the user's identity and the presented claims such as granted roles and tenant membership. Briefly, **authentication** reveals _who_ uses the service. In contrast, **authorization** controls _how_ the user can interact with the application's resources according to granted privileges. As the access control needs to rely on verified claims, authentication is a prerequisite to authorization. From perspective of CAP, the authentication method is freely customizable. For convenience, a set of authentication methods is supported out of the box to cover most common scenarios: - [XS User and Authentication and Authorization service](https://help.sap.com/docs/CP_AUTHORIZ_TRUST_MNG) (XSUAA) is a full-fleged [OAuth 2.0](https://oauth.net/2/) authorization server which allows to protect your endpoints in productive environments. JWT tokens issued by the server not only contain information about the user for authentication, but also assigned scopes and attributes for authorization. - [Identity Authentication Service](https://help.sap.com/docs/IDENTITY_AUTHENTICATION) (IAS) is an [OpenId Connect](https://openid.net/connect/) compliant service for next-generation identity and access management. As of today, CAP provides IAS authentication for incoming requests only. Authorization has to be explicitly managed by the application. - For _local development_ and _test_ scenario mock user authentication is provided as built-in feature. Find detailed instructions for setting up authentication in these runtime-specific guides: - [Set up authentication in Node.js.](/node.js/authentication) - [Set up authentication in Java.](/java/security#authentication) In _productive_ environment with security middleware activated, **all protocol adapter endpoints are authenticated by default**¹, even if no [restrictions](#restrictions) are configured. Multi-tenant SaaS-applications require authentication to provide tenant isolation out of the box. In case there is the business need to expose open endpoints for anonymous users, it's required to take extra measures depending on runtime and security middleware capabilities. > ¹ Starting with CAP Node.js 6.0.0 resp. CAP Java 1.25.0. _In previous versions endpoints without restrictions are public in single-tenant applications_. ### Defining Internal Services CDS services which are only meant for *internal* usage, shouldn't be exposed via protocol adapters. In order to prevent access from external clients, annotate those services with `@protocol: 'none'`: ```cds @protocol: 'none' service InternalService { ... } ``` The `InternalService` service can only receive events sent by in-process handlers. ## User Claims { #user-claims} CDS authorization is _model-driven_. This basically means that it binds access rules for CDS model elements to user claims. For instance, access to a service or entity is dependent on the role a user has been assigned to. Or you can even restrict access on an instance level, for example, to the user who created the instance.
The generic CDS authorization is built on a _CAP user concept_, which is an _abstraction_ of a concrete user type determined by the platform's identity service. This design decision makes different authentication strategies pluggable to generic CDS authorization.
After successful authentication, a (CAP) user is represented by the following properties: - Unique (logon) _name_ identifying the user. Unnamed users have a fixed name such as `system` or `anonymous`. - _Tenant_ for multitenant applications. - _Roles_ that the user has been granted by an administrator (see [User Roles](#roles)) or that are derived by the authentication level (see [Pseudo Roles](#pseudo-roles)). - _Attributes_ that the user has been assigned by an administrator. In the CDS model, some of the user properties can be referenced with the `$user` prefix: | User Property | Reference | |-------------------------------|---------------------| | Name | `$user` | | Tenant | `$user.tenant` | | Attribute (name \) | `$user.` | > A single user attribute can have several different values. For instance, the `$user.language` attribute can contain `['DE','FR']`. ### User Roles { #roles} As a basis for access control, you can design conceptual roles that are application specific. Such a role should reflect how a user can interact with the application. For instance, the role `Vendor` could describe users who are allowed to read sales articles and update sales figures. In contrast, a `ProcurementManager` can have full access to sales articles. Users can have several roles, that are assigned by an administrative user in the platform's authorization management solution. ::: tip CDS-based authorization deliberately refrains from using technical concepts, such as _scopes_ as in _OAuth_, in favor of user roles, which are closer to the conceptual domain of business applications. This also results in much **smaller JWT tokens**. ::: ### Pseudo Roles { #pseudo-roles} It's frequently required to define access rules that aren't based on an application-specific user role, but rather on the _authentication level_ of the request. For instance, a service could be accessible not only for identified, but also for anonymous (for example, unauthenticated) users. Such roles are called pseudo roles as they aren't assigned by user administrators, but are added at runtime automatically. The following predefined pseudo roles are currently supported by CAP: * `authenticated-user` refers to named or unnamed users who have presented a valid authentication claim such as a logon token. * [`system-user` denotes an unnamed user used for technical communication.](#system-user) * [`internal-user` is dedicated to distinguish application internal communication.](#internal-user) * `any` refers to all users including anonymous ones (that means, public access without authentication). #### system-user The pseudo role `system-user` allows you to separate access by _technical_ users from access by _business_ users. Note that the technical user can come from a SaaS or the PaaS tenant. Such technical user requests typically run in a _privileged_ mode without any restrictions on an instance level. For example, an action that implements a data replication into another system needs to access all entities of subscribed SaaS tenants and can’t be exposed to any business user. Note that `system-user` also implies `authenticated-user`. ::: tip For XSUAA or IAS authentication, the request user is attached with the pseudo role `system-user` if the presented JWT token has been issued with grant type `client_credentials` or `client_x509` for a trusted client application. ::: #### internal-user Pseudo-role `internal-user` allows to define application endpoints that can be accessed exclusively by the own PaaS tenant (technical communication). The advantage is that similar to `system-user` no technical CAP roles need to be defined to protect such internal endpoints. However, in contrast to `system-user`, the endpoints protected by this pseudo-role do not allow requests from any external technical clients. Hence is suitable for **technical intra-application communication**, see [Security > Application Zone](/guides/security/overview#application-zone). ::: tip For XSUAA or IAS authentication, the request user is attached with the pseudo role `internal-user` if the presented JWT token has been issued with grant type `client_credentials` or `client_x509` on basis of the **identical** XSUAA or IAS service instance. ::: ::: warning All technical clients that have access to the application's XSUAA or IAS service instance can call your service endpoints as `internal-user`. **Refrain from sharing this service instance with untrusted clients**, for instance by passing services keys or [SAP BTP Destination Service](https://help.sap.com/docs/connectivity/sap-btp-connectivity-cf/create-destinations-from-scratch) instances. ::: ### Mapping User Claims Depending on the configured [authentication](#prerequisite-authentication) strategy, CAP derives a *default set* of user claims containing the user's name, tenant and attributes: | CAP User Property | XSUAA JWT Property | IAS JWT Property | |---------------------|----------------------------------|-------------------------| | `$user` | `user_name` | `sub` | | `$user.tenant` | `zid` | `app_tid` | | `$user.` | `xs.user.attributes.` | All non-meta attributes | ::: tip CAP does not make any assumptions on the presented claims given in the token. String values are copied as they are. ::: In most cases, CAP's default mapping will match your requirements, but CAP also allows you to customize the mapping according to specific needs. For instance, `user_name` in XSUAA tokens is generally not unique if several customer IdPs are connected to the underlying identity service. Here a combination of `user_name` and `origin` mapped to `$user` might be a feasible solution that you implement in a custom adaptation. Similarly, attribute values can be normalized and prepared for [instance-based authorization](#instance-based-auth). Find details and examples how to programmatically redefine the user mapping here: - [Set up Authentication in Node.js.](/node.js/authentication) - [Custom Authentication in Java.](/java/security#custom-authentication) ::: warning Be very careful when redefining `$user` The user name is frequently stored with business data (for example, `managed` aspect) and might introduce migration efforts. Also consider data protection and privacy regulations when storing user data. ::: ## Restrictions { #restrictions} According to [authentication](#prerequisite-authentication), CAP endpoints are closed to anonymous users. But **by default, CDS services have no access control** which means that authenticated users are not restricted. To protect resources according to your business needs, you can define [restrictions](#restrict-annotation) that make the runtime enforce proper access control. Alternatively, you can add custom authorization logic by means of an [authorization enforcement API](#enforcement). Restrictions can be defined on *different CDS resources*: - Services - Entities - (Un)bound actions and functions You can influence the scope of a restriction by choosing an adequate hierarchy level in the CDS model. For instance, a restriction on the service level applies to all entities in the service. Additional restrictions on entities or actions can further limit authorized requests. See [combined restrictions](#combined-restrictions) for more details. Beside the scope, restrictions can limit access to resources with regards to *different dimensions*: - The [event](#restricting-events) of the request, that is, the type of the operation (what?) - The [roles](#roles) of the user (who?) - [Filter-condition](#instance-based-auth) on instances to operate on (which?) ### @readonly and @insertonly { #restricting-events} Annotate entities with `@readonly` or `@insertonly` to statically restrict allowed operations for **all** users as demonstrated in the example: ```cds service BookshopService { @readonly entity Books {...} @insertonly entity Orders {...} } ``` Note that both annotations introduce access control on an entity level. In contrast, for the sake of [input validation](/guides/providing-services#input-validation), you can also use `@readonly` on a property level. In addition, annotation `@Capabilities` from standard OData vocabulary is enforced by the runtimes analogously: ```cds service SomeService { @Capabilities: { InsertRestrictions.Insertable: true, UpdateRestrictions.Updatable: true, DeleteRestrictions.Deletable: false } entity Foo { key ID : UUID } } ``` #### Events to Auto-Exposed Entities { #events-and-auto-expose} In general, entities can be exposed in services in different ways: it can be **explicitly exposed** by the modeler (for example, by a projection), or it can be **auto-exposed** by the CDS compiler due to some reason. Access to auto-exposed entities needs to be controlled in a specific way. Consider the following example: ```cds context db { @cds.autoexpose entity Categories : cuid { // explicitly auto-exposed (by @cds.autoexpose) ... } entity Issues : cuid { // implicitly auto-exposed (by composition) category: Association to Categories; ... } entity Components : cuid { // explicitly exposed (by projection) issues: Composition of many Issues; ... } } service IssuesService { entity Components as projection on db.Components; } ``` As a result, the `IssuesService` service actually exposes *all* three entities from the `db` context: * `db.Components` is explicitly exposed due to the projection in the service. * `db.Issues` is implicitly auto-exposed by the compiler as it is a composition entity of `Components`. * `db.Categories` is explicitly auto-exposed due to the `@cds.autoexpose` annotation. In general, **implicitly auto-exposed entities cannot be accessed directly**, that means, only access via a navigation path (starting from an explicitly exposed entity) is allowed. In contrast, **explicitly auto-exposed entities can be accessed directly, but only as `@readonly`**. The rationale behind that is that entities representing value lists need to be readable at the service level, for instance to support value help lists. See details about `@cds.autoexpose` in [Auto-Exposed Entities](/guides/providing-services#auto-exposed-entities). This results in the following access matrix: | Request | `READ` | `WRITE` | |--------------------------------------------------------|:------:|:-------:| | `IssuesService.Components` | | | | `IssuesService.Issues` | | | | `IssuesService.Categories` | | | | `IssuesService.Components[].issues` | | | | `IssuesService.Components[].issues[].category` | | | ::: tip CodeLists such as `Languages`, `Currencies`, and `Countries` from `sap.common` are annotated with `@cds.autoexpose` and so are explicitly auto-exposed. ::: ### @requires { #requires} You can use the `@requires` annotation to control which (pseudo-)role a user requires to access a resource: ```cds annotate BrowseBooksService with @(requires: 'authenticated-user'); annotate ShopService.Books with @(requires: ['Vendor', 'ProcurementManager']); annotate ShopService.ReplicationAction with @(requires: 'system-user'); ``` In this example, the `BrowseBooksService` service is open for authenticated but not for anonymous users. A user who has the `Vendor` _or_ `ProcurementManager` role is allowed to access the `ShopService.Books` entity. Unbound action `ShopService.ReplicationAction` can only be triggered by a technical user. ::: tip When restricting service access through `@requires`, the service's metadata endpoints (that means, `/$metadata` as well as the service root `/`) are restricted by default as well. If you require public metadata, you can disable the check with [a custom express middleware](../../node.js/cds-serve#add-mw-pos) using the [privileged user](../../node.js/authentication#privileged-user) (Node.js) or through config cds.security.authentication.authenticateMetadataEndpoints = false (Java), respectively. Please be aware that the `/$metadata` endpoint is *not* checking for authorizations implied by `@restrict` annotation. ::: ### @restrict { #restrict-annotation} You can use the `@restrict` annotation to define authorizations on a fine-grained level. In essence, all kinds of restrictions that are based on static user roles, the request operation, and instance filters can be expressed by this annotation.
The building block of such a restriction is a single **privilege**, which has the general form: ```cds { grant:, to:, where: } ``` whereas the properties are: * `grant`: one or more events that the privilege applies to * `to`: one or more [user roles](#roles) that the privilege applies to (optional) * `where`: a filter condition that further restricts access on an instance level (optional). The following values are supported: - `grant` accepts all standard [CDS events](../../about/best-practices#events) (such as `READ`, `CREATE`, `UPDATE`, and `DELETE`) as well as action and function names. `WRITE` is a virtual event for all standard CDS events with write semantic (`CREATE`, `DELETE`, `UPDATE`, `UPSERT`) and `*` is a wildcard for all events. - The `to` property lists all [user roles](#roles) or [pseudo roles](#pseudo-roles) that the privilege applies to. Note that the `any` pseudo-role applies for all users and is the default if no value is provided. - The `where`-clause can contain a Boolean expression in [CQL](/cds/cql)-syntax that filters the instances that the event applies to. As it allows user values (name, attributes, etc.) and entity data as input, it's suitable for *dynamic authorizations based on the business domain*. Supported expressions and typical use cases are presented in [instance-based authorization](#instance-based-auth). A privilege is met, if and only if **all properties are fulfilled** for the current request. In the following example, orders can only be read by an `Auditor` who meets `AuditBy` element of the instance: ```cds entity Orders @(restrict: [ { grant: 'READ', to: 'Auditor', where: 'AuditBy = $user' } ]) {/*...*/} ``` If a privilege contains several events, only one of them needs to match the request event to comply with the privilege. The same holds, if there are multiple roles defined in the `to` property: ```cds entity Reviews @(restrict: [ { grant:['READ', 'WRITE'], to: ['Reviewer', 'Customer'] } ]) {/*...*/} ``` In this example, all users that have the `Reviewer` *or* `Customer` role can read *or* write to `Reviews`. You can build restrictions based on *multiple privileges*: ```cds entity Orders @(restrict: [ { grant: ['READ','WRITE'], to: 'Admin' }, { grant: 'READ', where: 'buyer = $user' } ]) {/*...*/} ``` A request passes such a restriction **if at least one of the privileges is met**. In this example, `Admin` users can read and write the `Orders` entity. But a user can also read all orders that have a `buyer` property that matches the request user. Similarly, the filter conditions of matched privileges are combined with logical OR: ```cds entity Orders @(restrict: [ { grant: 'READ', to: 'Auditor', where: 'country = $user.country' }, { grant: ['READ','WRITE'], where: 'CreatedBy = $user' }, ]) {/*...*/} ``` Here an `Auditor` user can read all orders with matching `country` or that they have created. > Annotations such as @requires or @readonly are just convenience shortcuts for @restrict, for example: - `@requires: 'Viewer'` is equivalent to `@restrict: [{grant:'*', to: 'Viewer'}]` - `@readonly` is the same as `@restrict: [{ grant:'READ' }]` Currently, the security annotations **are only evaluated on the target entity of the request**. Restrictions on associated entities touched by the operation aren't regarded. This has the following implications: - Restrictions of (recursively) expanded or inlined entities of a `READ` request aren't checked. - Deep inserts and updates are checked on the root entity only. See [solution sketches](#limitation-deep-authorization) for information about how to deal with that.{.learn-more} #### Supported Combinations with CDS Resources Restrictions can be defined on different types of CDS resources, but there are some limitations with regards to supported privileges: | CDS Resource | `grant` | `to` | `where` | Remark | |-----------------|:-------:|:----:|:-----------------:|---------------| | service | | | | = `@requires` | | entity | | | ¹ | | | action/function | | | ² | = `@requires` | > ¹For bound actions and functions that aren't bound against a collection, Node.js supports instance-based authorization at the entity level. For example, you can use `where` clauses that *contain references to the model*, such as `where: CreatedBy = $user`. For all bound actions and functions, Node.js supports simple static expressions at the entity level that *don't have any reference to the model*, such as `where: $user.level = 2`. > ² For unbound actions and functions, Node.js supports simple static expressions that *don't have any reference to the model*, such as `where: $user.level = 2`. Unsupported privilege properties are ignored by the runtime. Especially, for bound or unbound actions, the `grant` property is implicitly removed (assuming `grant: '*'` instead). The same also holds for functions: ```cds service CatalogService { entity Products as projection on db.Products { ... } actions { @(requires: 'Admin') action addRating (stars: Integer); } function getViewsCount @(restrict: [{ to: 'Admin' }]) () returns Integer; } ``` ### Combined Restrictions { #combined-restrictions} Restrictions can be defined on different levels in the CDS model hierarchy. Bound actions and functions refer to an entity, which in turn refers to a service. Unbound actions and functions refer directly to a service. As a general rule, **all authorization checks of the hierarchy need to be passed** (logical AND). This is illustrated in the following example: ```cds service CustomerService @(requires: 'authenticated-user') { entity Products @(restrict: [ { grant: 'READ' }, { grant: 'WRITE', to: 'Vendor' }, { grant: 'addRating', to: 'Customer'} ]) {/*...*/} actions { action addRating (stars: Integer); } entity Orders @(restrict: [ { grant: '*', to: 'Customer', where: 'CreatedBy = $user' } ]) {/*...*/} action monthlyBalance @(requires: 'Vendor') (); } ``` > The privilege for the `addRating` action is defined on an entity level. The resulting authorizations are illustrated in the following access matrix: | Operation | `Vendor` | `Customer` | `authenticated-user` | not authenticated | |--------------------------------------|:--------:|:----------------:|:--------------------:|-------------------| | `CustomerService.Products` (`READ`) | | | | | | `CustomerService.Products` (`WRITE`) | | | | | | `CustomerService.Products.addRating` | | | | | | `CustomerService.Orders` (*) | | ¹ | | | | `CustomerService.monthlyBalance` | | | | | > ¹ A `Vendor` user can only access the instances that they created.
The example models access rules for different roles in the same service. In general, this is _not recommended_ due to the high complexity. See [best practices](#dedicated-services) for information about how to avoid this. ### Draft Mode {#restrictions-and-draft-mode} Basically, the access control for entities in draft mode differs from the [general restriction rules](#restrict-annotation) that apply to (active) entities. A user, who has created a draft, should also be able to edit (`UPDATE`) or cancel the draft (`DELETE`). The following rules apply: - If a user has the privilege to create an entity (`CREATE`), he or she also has the privilege to create a **new** draft entity and update, delete, and activate it. - If a user has the privilege to update an entity (`UPDATE`), he or she also has the privilege to **put it into draft mode** and update, delete, and activate it. - Draft entities can only be edited by the creator user. + In the Node.js runtime, this includes calling bound actions/functions on the draft entity. ::: tip As a result of the derived authorization rules for draft entities, you don't need to take care of draft events when designing the CDS authorization model. ::: ### Auto-Exposed and Generated Entities { #autoexposed-restrictions} In general, **a service actually exposes more than the explicitly modeled entities from the CDS service model**. This stems from the fact that the compiler auto-exposes entities for the sake of completeness, for example, by adding composition entities. Another reason is generated entities for localization or draft support that need to appear in the service. Typically, such entities don't have restrictions. The emerging question is, how can requests to these entities be authorized? For illustration, let's extend the service `IssuesService` from [Events to Auto-Exposed Entities](#events-and-auto-expose) by adding a restriction to `Components`: ```cds annotate IssuesService.Components with @(restrict: [ { grant: '*', to: 'Supporter' }, { grant: 'READ', to: 'authenticated-user' } ]); ``` Basically, users with the `Supporter` role aren't restricted, whereas authenticated users can only read the `Components`. But what about the auto-exposed entities such as `IssuesService.Issues` and `IssuesService.Categories`? They could be a target of an (indirect) request as outlined in [Events to Auto-Exposed Entities](#events-and-auto-expose), but none of them are annotated with a concrete restriction. In general, the same also holds for service entities, which are generated by the compiler, for example, for localization or draft support. To close the gap with auto-exposed and generated entities, the authorization of such entities is delegated to a so-called **authorization entity**, which is the last entity in the request path, which bears authorization information, that means, which fulfills at least one of the following properties: - Explicitly exposed in the service - Annotated with a concrete restriction - Annotated with `@cds.autoexpose` So, the authorization for the requests in the example is delegated as follows: | Request Target | Authorization Entity | |--------------------------------------------------------|:--------------------------------------:| | `IssuesService.Components` | ¹ | | `IssuesService.Issues` | ¹ | | `IssuesService.Categories` | `IssuesService.Categories`² | | `IssuesService.Components[].issues` | `IssuesService.Components`³ | | `IssuesService.Components[].issues[].category` | `IssuesService.Categories`² | > ¹ Request is rejected.
> ² `@readonly` due to `@cds.autoexpose`
> ³ According to the restriction. `` is relevant for instance-based filters. ### Inheritance of Restrictions Service entities inherit the restriction from the database entity, on which they define a projection. An explicit restriction defined on a service entity *replaces* inherited restrictions from the underlying entity. Entity `Books` on a database level: ```cds namespace db; entity Books @(restrict: [ { grant: 'READ', to: 'Buyer' }, ]) {/*...*/} ``` Services `BuyerService` and `AdminService` on a service level: ```cds service BuyerService @(requires: 'authenticated-user'){ entity Books as projection on db.Books; /* inherits */ } service AdminService @(requires: 'authenticated-user'){ entity Books @(restrict: [ { grant: '*', to: 'Admin'} /* overrides */ ]) as projection on db.Books; } ``` | Events | `Buyer` | `Admin` | `authenticated-user` | |-------------------------------|:-------:|:-------:|:--------------------:| | `BuyerService.Books` (`READ`) | | | | | `AdminService.Books` (`*`) | | | | ::: tip We recommend defining restrictions on a database entity level only in exceptional cases. Inheritance and override mechanisms can lead to an unclear situation. ::: ::: warning _Warning_ A service level entity can't inherit a restriction with a `where` condition that doesn't match the projected entity. The restriction has to be overridden in this case. ::: ## Instance-Based Authorization { #instance-based-auth } The [restrict annotation](#restrict-annotation) for an entity allows you to enforce authorization checks that statically depend on the event type and user roles. In addition, you can define a `where`-condition that further limits the set of accessible instances. This condition, which acts like a filter, establishes an *instance-based authorization*. The condition defined in the `where`-clause typically associates domain data with static [user claims](#user-claims). Basically, it *either filters the result set in queries or accepts only write operations on instances that meet the condition*. This means that, the condition applies to following standard CDS events only¹: - `READ` (as result filter) - `UPDATE` (as reject condition²) - `DELETE` (as reject condition²) > ¹ Node.js supports _static expressions_ that *don't have any reference to the model* such as `where: $user.level = 2` for all events. > ² CAP Java uses a filter condition by default. For instance, a user is allowed to read or edit `Orders` (defined with the `managed` aspect) that they have created: ```cds annotate Orders with @(restrict: [ { grant: ['READ', 'UPDATE', 'DELETE'], where: 'CreatedBy = $user' } ]); ``` Or a `Vendor` can only edit articles on stock (that means `Articles.stock` positive): ```cds annotate Articles with @(restrict: [ { grant: ['UPDATE'], to: 'Vendor', where: 'stock > 0' } ]); ``` You can define `where`-conditions in restrictions based on [CQL](/cds/cql)-where-clauses.
Supported features are: * Predicates with arithmetic operators. * Combining predicates to expressions with `and` and `or` logical operators. * Value references to constants, [user attributes](#user-attrs), and entity data (elements including [paths](#association-paths)) * [Exists predicate](#exists-predicate) based on subselects.

CAP Java offers the option to enable rejection conditions for `UPDATE`, `DELETE` and custom events. Enable it using the configuration option cds.security.authorization.instance-based.reject-selected-unauthorized-entity.enabled: true.

::: info Avoid enumerable keys In case the filter condition is not met in an `UPDATE` or `DELETE` request, the runtime rejects the request (response code 403) even if the user is not even allowed to read the entity. To avoid to disclosure the existence of such entities to unauthorized users, make sure that the key is not efficiently enumerable. ::: ### User Attribute Values { #user-attrs} To refer to attribute values from the user claim, prefix the attribute name with '`$user.`' as outlined in [static user claims](#user-claims). For instance, `$user.country` refers to the attribute with the name `country`. In general, `$user.` contains a **list of attribute values** that are assigned to the user. The following rules apply: * A predicate in the `where` clause evaluates to `true` if one of the attribute values from the list matches the condition. * An empty (or not defined) list means that the user is fully restricted with regards to this attribute (that means that the predicate evaluates to `false`). For example, the condition `where: $user.country = countryCode` will grant a user with attribute values `country = ['DE', 'FR']` access to entity instances that have `countryCode = DE` _or_ `countryCode = FR`. In contrast, the user has no access to any entity instances if the value list of country is empty or the attribute is not available at all. #### Unrestricted XSUAA Attributes By default, all attributes defined in [XSUAA instances](#xsuaa-configuration) require a value (`valueRequired:true`) which is well-aligned with the CAP runtime that enforces restrictions on empty attributes. If you explicitly want to offer unrestricted attributes to customers, you need to do the following: 1. Switch your XSUAA configuration to `valueRequired:false` 2. Adjust the filter-condition accordingly, for example: `where: $user.country = countryCode or $user.country is null`. > If `$user.country` is undefined or empty, the overall expression evaluates to `true` reflecting the unrestricted attribute. ::: warning Refrain from unrestricted XSUAA attributes as they need to be designed very carefully as shown in the following example. ::: Consider this bad example with *unrestricted* attribute `country` (assuming `valueRequired:false` in XSUAA configuration): ```cds service SalesService @(requires: ['SalesAdmin', 'SalesManager']) { entity SalesOrgs @(restrict: [ { grant: '*', to: ['SalesAdmin', 'SalesManager'], where: '$user.country = countryCode or $user.country is null' } ]) { countryCode: String; /*...*/ } } ``` Let's assume a customer creates XSUAA roles `SalesManagerEMEA` with dedicated values (`['DE', 'FR', ...]`) and 'SalesAdmin' with *unrestricted* values. As expected, a user assigned only to 'SalesAdmin' has access to all `SalesOrgs`. But when role `SalesManagerEMEA` is added, *only* EMEA orgs are accessible suddenly! The preferred way is to model with restricted attribute `country` (`valueRequired:true`) and an additional grant: ```cds service SalesService @(requires: ['SalesAdmin', 'SalesManager']) { entity SalesOrgs @(restrict: [ { grant: '*', to: 'SalesManager', where: '$user.country = countryCode' }, { grant: '*', to: 'SalesAdmin' } ]) { countryCode: String; /*...*/ } } ``` ### Exists Predicate { #exists-predicate } In many cases, the authorization of an entity needs to be derived from entities reachable via association path. See [domain-driven authorization](#domain-driven-authorization) for more details. You can leverage the `exists` predicate in `where` conditions to define filters that directly apply to associated entities defined by an association path: ```cds service ProjectService @(requires: 'authenticated-user') { entity Projects @(restrict: [ { grant: ['READ', 'WRITE'], where: 'exists members[userId = $user and role = `Editor`]' } ]) { members: Association to many Members; /*...*/ } @readonly entity Members { key userId : User; key role: String enum { Viewer; Editor; }; /*...*/ } } ``` In the `ProjectService` example, only projects for which the current user is a member with role `Editor` are readable and editable. Note that with exception of the user ID (`$user`) **all authorization information originates from the business data**. Supported features of `exists` predicate: * Combine with other predicates in the `where` condition (`where: 'exists a1[...] or exists a2[...]`). * Define recursively (`where: 'exists a1[exists b1[...]]`). * Use target paths (`where: 'exists a1.b1[...]`). * Usage of [user attributes](#user-attrs). ::: warning Paths *inside* the filter (`where: 'exists a1[b1.c = ...]`) are not yet supported. ::: The following example demonstrates the last two features: ```cds service ProductsService @(requires: 'authenticated-user') { entity Products @(restrict: [ { grant: '*', where: 'exists producers.division[$user.division = name]'}]): cuid { producers : Association to many ProducingDivisions on producers.product = $self; } @readonly entity ProducingDivisions { key product : Association to Products; key division : Association to Divisions; } @readonly entity Divisions : cuid { name : String; producedProducts : Association to many ProducingDivisions on producedProducts.division = $self; } } ``` Here, the authorization of `Products` is derived from `Divisions` by leveraging the _n:m relationship_ via entity `ProducingDivisions`. Note that the path `producers.division` in the `exists` predicate points to target entity `Divisions`, where the filter with the user-dependent attribute `$user.division` is applied. ::: warning Consider Access Control Lists Be aware that deep paths might introduce a performance bottleneck. Access Control List (ACL) tables, managed by the application, allow efficient queries and might be the better option in this case. :::

### Association Paths { #association-paths} The `where`-condition in a restriction can also contain [CQL path expressions](/cds/cql#path-expressions) that navigate to elements of associated entities: ```cds service SalesOrderService @(requires: 'authenticated-user') { entity SalesOrders @(restrict: [ { grant: 'READ', where: 'product.productType = $user.productType' } ]) { product: Association to one Products; } entity Products { productType: String(32); /*...*/ } } ``` Paths on 1:n associations (`Association to many`) are only supported, _if the condition selects at most one associated instance_. It's highly recommended to use the [exists](#exists-predicate) predicate instead. ::: tip Be aware of increased execution time when modeling paths in the authorization check of frequently requested entities. Working with materialized views might be an option for performance improvement in this case. ::: ::: warning _Warning_ In Node.js association paths in `where`-clauses are currently only supported when using SAP HANA. ::: ## Best Practices CAP authorization allows you to control access to your business data on a fine granular level. But keep in mind that the high flexibility can end up in security vulnerabilities if not applied appropriately. In this perspective, lean and straightforward models are preferred. When modeling your access rules, the following recommendations can support you to design such models. ### Choose Conceptual Roles When defining user roles, one of the first options could be to align roles to the available _operations_ on entities, which results in roles such as `SalesOrders.Read`, `SalesOrders.Create`, `SalesOrders.Update`, and `SalesOrders.Delete`, etc. What is the problem with this approach? Think about the resulting number of roles that the user administrator has to handle when assigning them to business users. The administrator would also have to know the domain model precisely and understand the result of combining the roles. Similarly, assigning roles to operations only (`Read`, `Create`, `Update`, ...) typically doesn't fit your business needs.
We strongly recommend defining roles that describe **how a business user interacts with the system**. Roles like `Vendor`, `Customer`, or `Accountant` can be appropriate. With this approach, the application developers define the set of accessible resources in the CDS model for each role - and not the user administrator. ### Prefer Single-Purposed, Use-Case Specific Services { #dedicated-services} Have a closer look at this example: ```cds service CatalogService @(requires: 'authenticated-user') { entity Books @(restrict: [ { grant: 'READ' }, { grant: 'WRITE', to: 'Vendor', where: '$user.publishers = publisher' }, { grant: 'WRITE', to: 'Admin' } ]) as projection on db.Books; action doAccounting @(requires: ['Accountant', 'Admin']) (); } ``` Four different roles (`authenticated-user`, `Vendor`, `Accountant`, `Admin`) *share* the same service - `CatalogService`. As a result, it's confusing how a user can use `Books` or `doAccounting`. Considering the complexity of this small example (4 roles, 1 service, 2 resources), this approach can introduce a security risk, especially if the model is larger and subject to adaptation. Moreover, UIs defined for this service will likely appear unclear as well.
The fundamental purpose of services is to expose business data in a specific way. Hence, the more straightforward way is to **use a service for each of the roles**: ```cds @path:'browse' service CatalogService @(requires: 'authenticated-user') { @readonly entity Books as select from db.Books { title, publisher, price }; } @path:'internal' service VendorService @(requires: 'Vendor') { entity Books @(restrict: [ { grant: 'READ' }, { grant: 'WRITE', to: 'vendor', where: '$user.publishers = publisher' } ]) as projection on db.Books; } @path:'internal' service AccountantService @(requires: 'Accountant') { @readonly entity Books as projection on db.Books; action doAccounting(); } /*...*/ ``` ::: tip You can tailor the exposed data according to the corresponding role, even on the level of entity elements like in `CatalogService.Books`. ::: ### Prefer Dedicated Actions for Specific Use-Cases { #dedicated-actions} In some cases it can be helpful to restrict entity access as much as possible and create actions with dedicated restrictions for specific use cases, like in the following example: ```cds service GitHubRepositoryService @(requires: 'authenticated-user') { @readonly entity Organizations as projection on GitHub.Organizations actions { @(requires: 'Admin') action rename(newName : String); @(requires: 'Admin') action delete(); }; } ``` This service allows querying organizations for all authenticated users. In addition, `Admin` users are allowed to rename or delete. Granting `UPDATE` to `Admin` would allow administrators to change organization attributes that aren't meant to change. ### Think About Domain-Driven Authorization { #domain-driven-authorization} Static roles often don't fit into an intuitive authorization model. Instead of making authorization dependent from static properties of the user, it's often more appropriate to derive access rules from the business domain. For instance, all users assigned to a department (in the domain) are allowed to access the data of the organization comprising the department. Relationships in the entity model (for example, a department assignment to organization), influence authorization rules at runtime. In contrast to static user roles, **dynamic roles** are fully domain-driven. Revisit the [ProjectService example](#exists-predicate), which demonstrates how to leverage instance-based authorization to induce dynamic roles. Advantages of dynamic roles are: - The most flexible way to define authorizations - Induced authorizations according to business domain - Application-specific authorization model and intuitive UIs - Decentralized role management for application users (no central user administrator required) Drawbacks to be considered are: - Additional effort for modeling and designing application-specific role management (entities, services, UI) - Potentially higher security risk due to lower use of the framework functionality - Sharing authorization management with other (non-CAP) applications is harder to achieve - Dynamic role enforcement can introduce a performance penalty ### Control Exposure of Associations and Compositions { #limitation-deep-authorization} Note that exposed associations (and compositions) can disclose unauthorized data. Consider the following scenario: ```cds namespace db; entity Employees : cuid { // autoexposed! name: String(128); team: Association to Teams; contract: Composition of Contracts; } entity Contracts @(requires:'Manager') : cuid { // autoexposed! salary: Decimal; } entity Teams : cuid { members: Composition of many Employees on members.team = $self; } service ManageTeamsService @(requires:'Manager') { entity Teams as projection on db.Teams; } service BrowseEmployeesService @(requires:'Employee') { @readonly entity Teams as projection on db.Teams; // navigate to Contracts! } ``` A team (entity `Teams`) contains members of type `Employees`. An employee refers to a single contract (entity `Contracts`) which contains sensitive information that should be visible only to `Manager` users. `Employee` users should be able to browse the teams and their members, but aren't allowed to read or even edit their contracts.
As `db.Employees` and `db.Contracts` are auto-exposed, managers can navigate to all instances through the `ManageTeamsService.Teams` service entity (for example, OData request `/ManageTeamsService/Teams?$expand=members($expand=contract)`).
It's important to note that this also holds for an `Employee` user, as **only the target entity** `BrowseEmployeesService.Teams` **has to pass the authorization check in the generic handler, and not the associated entities**.
To solve this security issue, introduce a new service entity `BrowseEmployeesService.Employees` that removes the navigation to `Contracts` from the projection: ```cds service BrowseEmployeesService @(requires:'Employee') { @readonly entity Employees as projection on db.Employees excluding { contracts }; // hide contracts! @readonly entity Teams as projection on db.Teams; } ``` Now, an `Employee` user can't expand the contracts as the composition isn't reachable anymore from the service. ::: tip Associations without navigation links (for example, when an associated entity isn't exposed) are still critical with regards to security. ::: ### Design Authorization Models from the Start As shown before, defining an adequate authorization strategy has a deep impact on the service model. Apart from the fundamental decision, if you want to build your authorizations on [dynamic roles](#domain-driven-authorization), authorization requirements can result in rearranging service and entity definitions completely. In the worst case, this means rewriting huge parts of the application (including the UI). For this reason, it's *strongly* recommended to take security design into consideration at an early stage of your project. ### Keep it as Simple as Possible * If different authorizations are needed for different operations, it's easier to have them defined at the service level. If you start defining them at the entity level, all possible operations must be specified, otherwise the not mentioned operations are automatically forbidden. * If possible, try to define your authorizations either on the service or on the entity level. Mixing both variants increases complexity and not all combinations are supported either. ### Separation of Concerns Consider using [CDS Aspects](/cds/cdl#aspects) to separate the actual service definitions from authorization annotations as follows: ::: code-group ```cds [services.cds] service ReviewsService { /*...*/ } service CustomerService { entity Orders {/*...*/} entity Approval {/*...*/} } ``` ::: ::: code-group ```cds [services-auth.cds] service ReviewsService @(requires: 'authenticated-user'){ /*...*/ } service CustomerService @(requires: 'authenticated-user'){ entity Orders @(restrict: [ { grant: ['READ','WRITE'], to: 'admin' }, { grant: 'READ', where: 'buyer = $user' }, ]){/*...*/} entity Approval @(restrict: [ { grant: 'WRITE', where: '$user.level > 2' } ]){/*...*/} } ``` ::: This keeps your actual service definitions concise and focused on structure only. It also allows you to give authorization models separate ownership and lifecycle. ## Programmatic Enforcement { #enforcement} The service provider frameworks **automatically enforce** restrictions in generic handlers. They evaluate the annotations in the CDS models and, for example: * Reject incoming requests if static restrictions aren't met. * Add corresponding filters to queries for instance-based authorization, etc. If generic enforcement doesn't fit your needs, you can override or adapt it with **programmatic enforcement** in custom handlers: - [Authorization Enforcement in Node.js](/node.js/authentication#enforcement) - [Enforcement API & Custom Handlers in Java](/java/security#enforcement-api) ## Role Assignments with IAS and AMS The Authorization Management Service (AMS) as part of SAP Cloud Identity Services (SCI) provides libraries and services for developers of cloud business applications to declare, enforce and manage instance based authorization checks. When used together with CAP the AMS "Policies” can contain the CAP roles as well as additional filter criteria for instance based authorizations that can be defined in the CAP model. transformed to AMS policies and later on refined by customers user and authorization administrators in the SCI administration console and assigned to business users. ### Use AMS as Authorization Management System on SAP BTP SAP BTP is currently replacing the authorization management done with XSUAA by an integrated solution with AMS. AMS is integrated into SAP Cloud Identity (SCI), which will offer authentication, authorization, user provisioning and management in one place. For newly build applications the usage of AMS is generally recommended. The only constraint that comes with the usage of AMS is that customers need to copy their users to the Identity Directory Service as the central place to manage users for SAP BTP applications. This is also the general SAP strategy to simplify user management in the future. ### Case For XSUAA There is one use case where currently an XSUAA based authorization management is preferable: When XSUAA based services to be consumed by a CAP application come with their own business user roles and thus make user role assignment in the SAP Cloud Cockpit necessary. This will be resolved in the future when the authorization management will be fully based on the SCI Admin console. For example, SAP Task Center you want to consume an XSUAA-based service that requires own end user role. Apart from this, most services should be technical services that do not require an own authorization management that is not yet integrated in AMS. [Learn more about using IAS and AMS with CAP Node.js](https://github.com/SAP-samples/btp-developer-guide-cap/blob/main/documentation/xsuaa-to-ams/README.md){.learn-more} ## Role Assignments with XSUAA { #xsuaa-configuration} Information about roles and attributes has to be made available to the UAA platform service. This information enables the respective JWT tokens to be constructed and sent with the requests for authenticated users. In particular, the following happens automatically behind-the-scenes upon build: ### 1. Roles and Attributes Are Filled into the XSUAA Configuration Derive scopes, attributes, and role templates from the CDS model: ```sh cds add xsuaa --for production ``` This generates an _xs-security.json_ file: ::: code-group ```json [xs-security.json] { "scopes": [ { "name": "$XSAPPNAME.admin", "description": "admin" } ], "attributes": [ { "name": "level", "description": "level", "valueType": "s" } ], "role-templates": [ { "name": "admin", "scope-references": [ "$XSAPPNAME.admin" ], "description": "generated" } ] } ``` ::: For every role name in the CDS model, one scope and one role template are generated with the exact name of the CDS role. ::: tip Re-generate on model changes You can have such a file re-generated via ```sh cds compile srv --to xsuaa > xs-security.json ``` ::: See [Application Security Descriptor Configuration Syntax](https://help.sap.com/docs/HANA_CLOUD_DATABASE/b9902c314aef4afb8f7a29bf8c5b37b3/6d3ed64092f748cbac691abc5fe52985.html) in the SAP HANA Platform documentation for the syntax of the _xs-security.json_ and advanced configuration options. ::: warning Avoid invalid characters in your models Roles modeled in CDS may contain characters considered invalid by the XSUAA service. ::: If you modify the _xs-security.json_ manually, make sure that the scope names in the file exactly match the role names in the CDS model, as these scope names will be checked at runtime. ### 2. XSUAA Configuration Is Completed and Published #### Through MTA Build If there's no _mta.yaml_ present, run this command: ```sh cds add mta ``` ::: details See what this does in the background… 1. It creates an _mta.yaml_ file with an `xsuaa` service. 2. The created service added to the `requires` section of your backend, and possibly other services requiring authentication. ::: code-group ```yaml [mta.yaml] modules: - name: bookshop-srv requires: - bookshop-auth // [!code ++] resources: name: bookshop-auth // [!code ++] type: org.cloudfoundry.managed-service // [!code ++] parameters: // [!code ++] service: xsuaa // [!code ++] service-plan: application // [!code ++] path: ./xs-security.json # include cds managed scopes and role templates // [!code ++] config: // [!code ++] xsappname: bookshop-${org}-${space} // [!code ++] tenant-mode: dedicated # 'shared' for multitenant deployments // [!code ++] ``` ::: Inline configuration in the _mta.yaml_ `config` block and the _xs-security.json_ file are merged. If there are conflicts, the [MTA security configuration](https://help.sap.com/docs/HANA_CLOUD_DATABASE/b9902c314aef4afb8f7a29bf8c5b37b3/6d3ed64092f748cbac691abc5fe52985.html) has priority. [Learn more about **building and deploying MTA applications**.](/guides/deployment/){ .learn-more} ### 3. Assembling Roles and Assigning Roles to Users This is a manual step an administrator would do in SAP BTP Cockpit. See [Set Up the Roles for the Application](/node.js/authentication#auth-in-cockpit) for more details. If a user attribute isn't set for a user in the IdP of the SAP BTP Cockpit, this means that the user has no restriction for this attribute. For example, if a user has no value set for an attribute "Country", they're allowed to see data records for all countries. In the _xs-security.json_, the `attribute` entity has a property `valueRequired` where the developer can specify whether unrestricted access is possible by not assigning a value to the attribute. ### 4. Scopes Are Narrowed to Local Roles Based on this, the JWT token for an administrator contains a scope `my.app.admin`. From within service implementations of `my.app` you can reference the scope: ```js req.user.is ("admin") ``` ... and, if necessary, from others by: ```js req.user.is ("my.app.admin") ```
> See the following sections for more details: - [Developing Security Artifacts in SAP BTP](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/419ae2ef1ddd49dca9eb65af2d67c6ec.html) - [Maintaining Application Security in XS Advanced](https://help.sap.com/docs/HANA_CLOUD_DATABASE/b9902c314aef4afb8f7a29bf8c5b37b3/35d910ee7c7a445a950b6aad989a5a26.html) # Platform Security This section provides an overview about the security architecture of CAP applications on different platforms. ## Platform Compliance { #platform-compliance } CAP applications run in a certain environment, that is, in the context of some platform framework that has specific characteristics. The underlying framework has a major impact on the security of the application, regardless of whether it runs a [cloud](#cloud) environment or [local](#local) environment. Moreover, CAP applications are tightly integrated with [platform services](#btp-services), in particular with identity and persistence service. ::: warning End-to-end security necessarily requires compliance with all security policies of all involved components. CAP application security requires consistent security configuration of the underlying platform and all consumed services. Consult the relevant security documentation accordingly. ::: ### CAP in Cloud Environment { #cloud } Currently, CAP supports to run on two cloud runtimes of [SAP Business Technology Platform](https://help.sap.com/docs/btp): - [SAP BTP, Cloud Foundry Runtime](https://help.sap.com/docs/btp/sap-business-technology-platform/cloud-foundry-environment) - [SAP BTP, Kyma Runtime](https://help.sap.com/docs/btp/sap-business-technology-platform/kyma-environment) Application providers are responsible to ensure a **secure platform environment**. In particular, this includes *configuring* [platform services](#btp-services) the application consumes. For instance, the provider (user) administrator needs to configure the [identity service](#identity-service) to separate platform users from business users that come from different identity providers. Likewise login policies (for example, multifactor authentication or single-sign-on) need to be aligned with company-specific requirements. Note, that achieving production-ready security requires to meet all relevant aspects of the **development process** as well. For instance, source code repositories need to be protected and may not contain any secrets or personal data. Likewise, the **deployment process** needs to be secured. That includes not only setting up CI/CD pipelines running on technical platform users, but also defining integration tests to ensure properly secured application endpoints. As part of **secure operations**, application providers need to establish a patch and vulnerability management, as well as a secure support process. For example, component versions need to be updated and credentials need to be rotated regularly. ::: warning The application provider is responsible to **develop, deploy, and operate the application in a secure platform environment**. CAP offers seamless integration into platform services and tools to help to meet these requirements. ::: Find more about BTP platform security here: [SAP BTP Security](https://help.sap.com/docs/btp/sap-business-technology-platform/security-e129aa20c78c4a9fb379b9803b02e5f6){.learn-more} [SAP BTP Security Recommendations](https://help.sap.com/docs/btp/sap-btp-security-recommendations-c8a9bb59fe624f0981efa0eff2497d7d/sap-btp-security-recommendations){.learn-more} [SAP BTP Security (Community)](https://pages.community.sap.com/topics/btp-security){.learn-more}

### CAP in Local Environment { #local } Security not only plays a crucial role in [cloud](#cloud) environments, but also during local development. Apparently the security requirements are different from cloud scenario as local endpoints are typically not exposed for remote clients. But there are still a few things to consider because exploited vulnerabilities could be the basis for attacks on productive cloud services: - Make sure that locally started HTTP endpoints are bound to `localhost`. - In case you run your service in hybrid mode with bindings to cloud service instances, use [cds bind](../../advanced/hybrid-testing) instead of copying bindings manually to `default-env.json` file. `cds bind` avoids materialization of secrets to local disc, which is inherently dangerous. - Don't write sensitive data to application logs, also not via debug logging. - Don't test with real business data, for example, copied from a productive system. ### SAP BTP Services for Security { #btp-services} SAP BTP provides a range of platform services that your CAP applications can utilize to meet production-grade security requirements. To ensure the security of your CAP applications, it's crucial to comply with the service level agreement (SLA) of these platform services. *As the provider of the application, you play a key role in meeting these requirements by correctly configuring and using these services.* ::: tip SAP BTP services and the underlying platform infrastructure hold various certifications and attestations, which can be found under the naming of SAP Cloud Platform in the [SAP Trust Center](https://www.sap.com/about/trust-center/certification-compliance/compliance-finder.html?search=SAP%20Business%20Technology%20Platform%20ISO). ::: The CAP framework offers flexible APIs that you can integrate with various services, including your custom services. If you replace platform services with your custom ones, it's important to ensure that the service level agreements (SLAs) CAP depends on are still met. The most important services for security offered by the platform: [Webcast SAP BTP Cloud Identity and Security Services](https://assets.dm.ux.sap.com/webinars/sap-user-groups-k4u/pdfs/221117_sap_security_webcast_series_sap_btp_cloud_identity_and_security_services.pdf){.learn-more} #### [SAP Cloud Identity Services - Identity Authentication](https://help.sap.com/docs/IDENTITY_AUTHENTICATION) { #identity-service } The Identity Authentication service defines the user base for (CAP) applications and services, and allows to control access. Customers can integrate their 3rd party or on-premise identity provider (IdP) and harden security by defining multifactor authentication or by narrowing client IP ranges. This service helps to introduce a strict separation between platform users (provider) and business users (subscribers), a requirement of CAP. It supports various authentication methods, including SAML 2.0 and [OpenID Connect](https://openid.net/connect/), and allows for the configuration of single sign-on access. [Learn more in the security guide.](https://help.sap.com/docs/IDENTITY_AUTHENTICATION?#discover_task-security){.learn-more} #### [SAP Authorization and Trust Management Service](https://help.sap.com/docs/CP_AUTHORIZ_TRUST_MNG) The service lets customers manage user authorizations in technical roles at application level, which can be aggregated into business-level role collections for large-scale cloud scenarios. Obviously, developers must define application roles carefully as they form basic access rules to business data. #### [SAP Malware Scanning Service](https://help.sap.com/docs/MALWARE_SCANNING) This service can be used to scan transferred business documents for malware and viruses. Currently, there is no CAP integration. A scan needs to be triggered by the business application explicitly. [Learn more in the security guide.](https://help.sap.com/docs/btp?#operate_task-security){.learn-more} #### [SAP Credential Store](https://help.sap.com/docs/CREDENTIAL_STORE) Credentials managed by applications need to be stored in a secure way. This service provides a REST API for (CAP) applications to store and retrieve credentials at runtime. [Learn more in the security guide.](https://help.sap.com/docs/CREDENTIAL_STORE?#discover_task-security){.learn-more} #### [SAP BTP Connectivity](https://help.sap.com/docs/CP_CONNECTIVITY) The connectivity service allows SAP BTP applications to securely access remote services that run on the Internet or on-premise. It provides a way to establish a secure communication channel between remote endpoints that are connected via an untrusted network infrastructure. [Learn more in the security guide.](https://help.sap.com/docs/CP_CONNECTIVITY/cca91383641e40ffbe03bdc78f00f681/cb50b6191615478aa11d2050dada467d.html){.learn-more} ## Architecture and Platform Requirements As [pointed out](#platform-compliance), CAP cloud applications run in a specific context that has a major impact on the security [architecture](#architecture-overview). CAP requires a dedicated [platform environment](#platform-environment) to integrate with, in order to ensure end-to-end security. ### Architecture Overview { #architecture-overview } The following diagram provides a high-level overview about the security-relevant aspects of a deployed CAP application in a cloud environment: ![This TAM graphic is explained in the accompanying text.](./assets/cap-security-architecture-overview.png){} To serve a business request, different runtime components are involved: a request, issued by a UI or technical client ([public zone](#public-zone)), is forwarded by a gateway or ingress router to the CAP application. In case of a UI request, an [Application Router](https://help.sap.com/docs/btp/sap-business-technology-platform/application-router) instance acts as a proxy. The CAP application might make use of a CAP sidecar. All application components ([application zone](#application-zone)) might make use of platform services such as database or identity service ([platform zone](#platform-zone)). #### Public Zone { #public-zone } From CAP's point of view, all components without specific security requirements belong to the public zone. Therefore, you shouldn't rely on the behavior or structure of consumer components like browsers or technical clients for the security of server components. The platform's gateway provides a single point of entry for any incoming call and defines the API visible to the public zone. As malicious users have free access to the public zone, these endpoints need to be protected carefully. Ideally, you should limit the number of exposed endpoints to a minimum, perhaps through proper network configuration. #### Platform Zone { #platform-zone } The platform zone contains all platform components and services that are *configured and maintained* by the application provider. CAP applications consume these low-level [platform services](#btp-services) to handle more complex business requests. For instance, persistence service to store business data and identity service to authenticate the business user play a fundamental role. The platform zone also includes the gateway, which is the main entry point for external requests. Additionally, it may contain extra ingress routers. #### Application Zone { #application-zone} The application zone comprises all microservices that represent a CAP application. They are tightly integrated and form a unit of trust. The application provider is responsible to *develop, deploy and operate* these services: - The [Application Router](https://help.sap.com/docs/btp/sap-business-technology-platform/application-router) acts as as an optional reverse proxy wrapping the application service and providing business-independent functionality required for UIs. This includes serving UI content, providing a login flow as well as managing the session with the browser. It can be deployed as application (reusable module) or alternatively consumed as a [service](https://help.sap.com/docs/btp/sap-business-technology-platform/managed-application-router). - The CAP application service exposes the API to serve business requests. Usually, it makes use of lower-level platform services. As built on CAP, a significant number of security requirements is covered either out of the box or by adding minimal configuration. - The optional CAP sidecar (reusable module) is used to outsource application-independent tasks such as providing multitenancy and extension support. Application providers, that is platform users, have privileged access to the application zone. In contrast, application subscribers, that is business users, are restricted to a minimal interface. ::: warning ❗ Application providers **may not share any secrets from the application zone** such as binding information with other components or persons. In a productive environment, it is recommended to deploy and operate the application on behalf of a technical user. ::: ::: tip Without limitation of generality, there may be multiple CAP services or sidecars according to common [microservice architecture pattern](https://microservices.io/patterns/microservices.html). ::: ### Required Platform Environment { #platform-environment } There are several assumptions that a CAP application needs to make about the platform environment it is deployed to: 1. Application and (platform) service endpoints are exposed externally by the API gateway via TLS protocol. Hence, the **CAP application can offer a pure HTTP endpoint** without having to enforce TLS and to deal with certificates. 2. The server certificates presented by the external endpoints are signed by a trusted certificate authority. This **frees CAP applications from the need to manage trust certificates**. The underlying runtimes (Java or Node) can validate the server certificates by default. 3. **Secrets** that are required to protect the application or to consume other platform services **are injected by the platform** into the application in a secure way. All supported [environments](overview#cloud) fulfill the given requirements. Additional requirements could be added in future. ::: tip Custom domain certificates need to be signed by trusted certificate authority. ::: ::: warning ❗ **In general, application endpoints are visible to public zone**. Hence, CAP can't rely on private endpoints. In particular, an application router does not prevent external access to the CAP application service. As a consequence, **all CAP endpoints must be protected in an appropriate manner**. ::: # Security Aspects This section describes in detail what CAP offers to protect your application. ## Secure Communications { #secure-communications } ### Encrypted Communication Channels { #encrypted-channels } *Integrity* and *confidentiality* of data being transferred between any communication endpoints needs to be guaranteed. In particular, this holds true for communication between client and server ([public zone](./overview#public-zone) resp. [platform zone](./overview#platform-zone)), but also for service-to-service communication (within a platform zone). That means the communication channels are established in a way that rules out undetected data manipulation or disclosure. #### Inbound Communication (Server) { #inbound } [SAP BTP](https://help.sap.com/docs/btp/sap-business-technology-platform/btp-security) exclusively establishes encrypted communication channels based on HTTPS/TLS as shown in the [architecture overview](./overview#architecture-overview) and hence fulfills the requirements out of the box. For all deployed (CAP) applications and platform services, the platform's API gateway resp. ingress router provides TLS endpoints accepting incoming request and forwards to the backing services via HTTP. The HTTP endpoints of microservices are only accessible for the router in terms of network technology (perimeter security) and therefore aren't visible for clients in public and platform zone. Likewise microservices can only serve a single network port, which the platform has opened for the hosting container. The router endpoints are configured with an up to date TLS protocol version containing a state-of-the-art cipher suite. Server authentication is given by X.509 server certificates signed by a trusted certificate authority. ::: tip It's mandatory for public clients to authenticate the server and to verify the server's identity by matching the target host name with the host name in the server certificate. ::: ::: tip Manually provided certificates for [custom domains](https://help.sap.com/docs/CUSTOM_DOMAINS/6f35a23466ee4df0b19085c9c52f9c29/4f4c3ff62fd2413089dce8a973620167.html) need to be signed by a [trusted certificate authority](https://help.sap.com/docs/btp/sap-business-technology-platform/trusted-certificate-authentication). ::: #### Outbound Communication (Client) { #outbound } As platform services and other applications deployed to BTP are only accessible via exposed TLS router endpoints, outbound connections are automatically secured as well. Consequently, technical clients have to [validate the server certificate](#inbound) for proper server authentication. Also here CAP application developers don't need to deal with HTTPS/TLS connection setup provided the client code is build on CAP offerings such as HANA Cloud Service or CloudSDK integration. ::: warning The **CAP application needs to ensure adequate protection of secrets** that are injected into CAP microservices, for example: - [mTLS authentication is enabled](https://help.sap.com/docs/btp/sap-business-technology-platform/enable-mtls-authentication-to-sap-authorization-and-trust-management-service-for-your-application) in the XSUAA service instance of your application and also for XSUAA reuse instances of platform services. - Ensure that [service bindings and keys](https://help.sap.com/docs/btp/sap-business-technology-platform/using-services-in-cloud-foundry-environment) aren't compromised (rotate regularly). - SAP BTP Connectivity services are maintained [securely](https://help.sap.com/docs/connectivity/sap-btp-connectivity-cf/connectivity-security). ::: #### Internal Communication (Client and Server) { #internal } Depending on the target platform, closely coupled microservices of the application zone might also communicate via trusted network channels instead of using [outbound connections](#outbound). For instance, a CAP service could communicate to a CAP sidecar, which is deployed to the same container via localhost HTTP connection. ::: tip CAP allows to use alternative communication channels, but application operators are responsible to set up them in a secure manner. ::: ::: tip CAP applications don't have to deal with TLS, communication encryption, or certificates, for inbound as well as outbound connections. ::: ### Filtering Internet Traffic { #filtering } Reducing attack surface by filtering communication from or to public zone increases the overall security protection level. By default, the platform comes with a standard set of services and configurations to protect network communication building on security features of the underlying hyperscaler. ::: warning Measures to further **restrict web access to your application** can be applied at platform level and aren't offered by CAP. For instance, [CF Route service](https://docs.cloudfoundry.org/services/route-services.html) can be used to implement route-specific restriction rules. ::: ## Secure Authentication { #secure-authentication } None-public resources may only be accessed by authenticated users. Hence, authentication plays a key role for product security on different levels: - **Business users** consume the application via web interface. In multitenant applications, they come from different subscriber tenants that need to be isolated from each other. - **Platform users** operate the application and have privileged access to its components on OS level (containers, configurations, logs etc.). Platform users come from the provider tenant. Managing user pools, providing a logon flow, and processing authentication are complex and highly security-critical tasks **that shouldn't be tackled by applications**. Instead, applications should rely on an identity service provided by the platform which is [seamlessly integrated by CAP](#authenticate-requests). Find more about platform and business users: [SAP BTP User and Member Management](https://help.sap.com/docs/btp/sap-business-technology-platform/user-and-member-management){.learn-more} ### Server Requests { #authenticate-requests } SAP BTP offers central identity services [SAP Cloud Identity Services - Identity Authentication](https://help.sap.com/docs/IDENTITY_AUTHENTICATION) resp. [SAP Authorization and Trust Management Service](https://help.sap.com/docs/CP_AUTHORIZ_TRUST_MNG) for managing and authenticating platform and business users providing: - User authentication flows (OpenID connect), for example, multifactor authentication - Federation of custom identity providers (IdPs) - Single-sign on - Principal propagation - Password and session policies etc. The central platform service provides applications with a large set of industry-proven security features, which is why applications don't have to develop their own extensions and run the risk of security flaws. CAP doesn't require any specific authentication strategy, but it provides out of the box integration with the platform identity service. On configured authentication, *all CAP endpoints are authenticated by default*. ::: warning ❗ **CAP applications need to ensure that an appropriate [authentication method](/guides/security/authorization#prerequisite-authentication) is configured**. It's highly recommended to establish integration tests to safeguard a valid configuration. ::: Learn more about user model and identity providers here: [SAP BTP Security](https://help.sap.com/docs/btp/sap-business-technology-platform/btp-security){.learn-more} ### Remote Services { #authenticate-remote } CAP microservices consume remote services and hence need to be authenticated as technical client as well. Similar to [request authentication](#authenticate-requests), CAP saves applications from having to implement secure setup of service to service communication: - CAP interacts with platform services such as [Event Mesh](../messaging/) or [SaaS Provisioning Service](../deployment/to-cf) on basis of platform-injected service bindings. - CAP offers consumption of [Remote Services](../using-services) on basis of [SAP BTP destinations](../using-services#btp-destinations). Note that the applied authentication strategy is specified by server offering and resp. configuration and not limited by CAP.

### Maintaining Sessions { #sessions } CAP microservices require [authentication](#authenticate-requests) of all requests, but they don't support logon flows for UI clients. Being stateless, they neither establish a session with the client to store login information such as an OAuth 2 token that needs to be passed in each server request. To close this gap, UI-based CAP applications can use an [Application Router](https://help.sap.com/docs/btp/sap-business-technology-platform/application-router) instance or service as reverse proxy as depicted in the [diagram](./overview#architecture-overview). The Application Router redirects the login to the identity service, fetches an OAuth2 token, and stores it into a secure session cookie. ::: warning ❗ The **Application Router endpoints don't hide CAP endpoints** in the service backend. Hence, authentication is still mandatory for CAP microservices. ::: ### Maintaining Secrets { #secrets } To run a CAP application that authenticates users and consumes remote services, **it isn't required to manage any secrets such as keys, tokens, or passwords**. Also CAP doesn't store any of them, but relies on platform [injection mechanisms](./overview#platform-environment) or [destinations](../using-services#btp-destinations). ::: tip In case you still need to store any secrets, use a platform service [SAP Credential Store](https://help.sap.com/docs/CREDENTIAL_STORE). ::: ## Secure Authorization { #secure-authorization } According to segregation of duties paradigm, user administrators need to control how different users may interact with the application. Critical combinations of authorizations must be avoided. Basically, access rules for [business users](#business-authz) are different from [platform users](#platform-authz). ### Business Users { #business-authz } To align with the principle of least privilege, applications need to enforce fine-grained access control for business users from the subscriber tenants. Depending from the business scenario, users need to be restricted to operations they perform on server resources, for example, reading an entity collection. Moreover, they might also be limited to a subset of data entries, that is, they may only operate on a filtered view on the data. The set of rules that apply to a user reflects a specific conceptual role that describes the interaction with the application to fulfill a business scenario. Obviously, the business roles are dependent from the scenarios and hence *need to be defined by the application developers*. Enforcing authorization rules at runtime is highly security-critical and shouldn't be implemented by the application as this would introduce the risk of security flaws. Instead, [CAP authorizations](/guides/security/authorization) follow a declarative approach allowing applications to design comprehensive access rules in the CDS model. Resources in the model such as services or entities can be restricted to users that fulfill specific conditions as declared in `@requires` or `@restrict` [annotations](/guides/security/authorization#restrictions). According to the declarations, server-side authorization enforcement is guaranteed for all requests. It's executed close before accessing the corresponding resources. ::: warning ❗ **By default, CAP services and entities aren't authorized**. Application developers need to **design and test access rules** according to the business need. ::: ::: tip To verify CAP authorizations in your model, it's recommended to use [CDS lint rules](../../tools/cds-lint/rules/). ::: The rules prepared by application developers are applied to business users according to grants given by the subscribers user administrator, that is, they're applied tenant-specific. CAP authorizations can be defined dependently from [user claims](/guides/security/authorization#user-claims) such as [XSUAA scopes or attributes](https://help.sap.com/docs/btp/sap-business-technology-platform/application-security-descriptor-configuration-syntax) that are deployed by application developers and granted by the user administrator of the subscriber. Hence, CAP provides a seamless integration of central identity service without technical lock-in. ::: tip You can generate the `xs-security.json` [descriptor file](https://help.sap.com/docs/btp/sap-business-technology-platform/protecting-your-application) of the application's XSUAA instance by executing `cds add xsuaa` in the project root folder. The XSUAA scopes, roles, and attributes are derived from the CAP authorization model. ::: ::: warning CAP authorization enforcement doesn't automatically log successful and unsuccessful authorization checks. Applications need to add corresponding custom handlers to support it. ::: #### Authorization of CAP Endpoints { #cap-endpoints } In general, responses created by *standard* CAP handlers and services are created on need-to-know basis. This means, authorized users only receive server information according to their privilege. Therefore, business users won't gain information about server host names, any version of application server component, generated queries etc. Based on the CDS model and configuration of CDS services, the CAP runtime exposes following endpoints: | Name | Configuration | URL | Authorization | |-------------------|------------------|-------------------------------------------|-----------------------------------------------| | CDS Service `Foo` | `service Foo {}` | `//Foo/**`¹ | `@restrict`/`@requires`² | | | OData v2/v4 | `//Foo/$metadata`¹ | See [here](/guides/security/authorization#requires) | | Index page | | `/index.html` | none, but disabled in production | > ¹ See [protocols and paths](../../java/cqn-services/application-services#configure-path-and-protocol) > ² No authorization by default Based on configured features, the CAP runtime exposes additional callback endpoints for specific platform service:

| Platform service | URL | Authorization | |------------------------------|-----------------------------|--------------------------------------------------------------------------------------------------------| | Multitenancy (SaaS Registry) | `/mt/v1.0/subscriptions/**` | Technical role `mtcallback` |

| Platform service | URL | Authorization | |------------------------------|-------------------------|---------------| | Multitenancy (SaaS Registry) | none so far for Node.js | |

Moreover, technical [MTXs CAP services](../multitenancy/mtxs) may be configured, for example, as sidecar microservice to support higher-level features such as Feature Toggles or Multitenancy: | CAP service | URL | Authorization | ----------- | --- | ------------- | [cds.xt.ModelProviderService](../multitenancy/mtxs#modelproviderservice) | `/-/cds/model-provider/**` | Internal, technical user¹ | [cds.xt.DeploymentService](../multitenancy/mtxs#deploymentservice) | `/-/cds/deployment/**` | | Internal, technical user¹, or technical role `cds.Subscriber` | [cds.xt.SaasProvisioningService](../multitenancy/mtxs#saasprovisioningservice) | `/-/cds/saas-provisioning/**` | Internal, technical user¹, or technical roles `cds.Subscriber` resp. `mtcallback` | [cds.xt.ExtensibilityService](../multitenancy/mtxs#extensibilityservice) | `/-/cds/extensibility/**` | Internal, technical user¹, or technical roles `cds.ExtensionDeveloper` > ¹ The microservice running the MTXS CAP service needs to be deployed to the [application zone](./overview#application-zone) and hence has established trust with the CAP application client, for instance given by shared XSUAA instance. Authentication for a CAP sidecar needs to be configured just like any other CAP application. ::: warning ❗ Ensure that technical roles such as `cds.Subscriber`, `mtcallback`, or `emcallback` **are never included in business roles**. ::: ### Platform Users { #platform-authz } Similar to [business consumption](#business-authz), different scenarios apply on operator level that need to be separated by dedicated access rules: deployment resp. configuration, monitoring, support, audit logs etc. *CAP doesn't cover authorization of platform users*. Please refer to security documentation of the underlying SAP BTP runtime environment: - [Roles in the Cloud Foundry Environment](https://help.sap.com/docs/btp/sap-business-technology-platform/about-roles-in-cloud-foundry-environment) - [Roles in the Kyma Environment](https://help.sap.com/docs/btp/sap-business-technology-platform/assign-roles-in-kyma-environment) ## Secure Multi-Tenancy { #secure-multitenancy } Multitenant SaaS-applications need to take care for security aspects on a higher level. Different subscriber tenants share the same runtime stack to interact with the CAP application. Ideally, from perspective of a single tenant, the runtime should look like a self-contained virtual system that doesn't interfere with any other tenant. All directly or indirectly involved services that process the business request require to isolate with regards to several dimensions: - No breakout to [persisted data](#isolated-persistent-data) - No breakout to [transient data](#isolated-transient-data) - Limited [resource consumption](#limiting-resource-consumption) The CAP runtime is designed from scratch to support tenant isolation: ### Isolated Persistent Data { #isolated-persistent-data } Having configured [Multitenancy in CAP](../multitenancy/), when serving a business request, CAP automatically targets an isolated HDI container dedicated for the request tenant to execute DB statements. Here, CAP's data query API based on [CQN](../../cds/cqn) is orthogonal to multitenancy, that is, custom CAP handlers can be implemented agnostic to MT. During tenant onboarding process, CAP triggers the HDI container creation via [SAP HANA Cloud Services](https://help.sap.com/docs/HANA_SERVICE_CF/cc53ad464a57404b8d453bbadbc81ceb/f70399be7fca4508aa0e33e138dbd84d.html). The containers have separated DB schemas and dedicated technical DB users for access. CAP guarantees that code for business requests runs on a DB connection opened for the technical user of the tenant's container. ### Isolated Transient Data { #isolated-transient-data } Although CAP microservices are stateless, the CAP Java runtime (generic handlers inclusive) needs to cache data in-memory for performance reasons. For instance, filters for [instance-based authorization](/guides/security/authorization#instance-based-auth) are constructed only once and are reused in subsequent requests.

To minimize risk of a data breach by exposing transient data at runtime, the CAP Java runtime explicitly refrains from declaring and using static mutable objects in Java heap. Instead, request-related data such as the [EventContext](https://www.javadoc.io/doc/com.sap.cds/cds-services-api/latest/com/sap/cds/services/EventContext.html) is provided via thread-local storage. Likewise, data is stored in tenant-maps that are transitively referenced by the [CdsRuntime](https://www.javadoc.io/doc/com.sap.cds/cds-services-api/latest/com/sap/cds/services/CdsRuntime.html) instance. ::: warning Make sure that custom code doesn't break tenant data isolation. :::

Request-related data is propagated down the call stack via the continuation-local variable [cds.context](../../node.js/events#cds-context). ::: warning Make sure that custom code doesn't break tenant data isolation or leak data across concurrent requests. ::: As a best practice, you should not put any non-static variables in the closures of your service implementations. ##### **Bad example:** {.bad} ::: code-group ```js [srv/cat-service.js] module.exports = srv => { let books // <- leaks data across tenants and concurrent requests // [!code error] srv.on('READ', 'Books', async function(req, next) { if (books) return books return books = await next() }) } ``` :::

### Limiting Resource Consumption { #limiting-resource-consumption } Tenant-aware microservices also need to handle resource consumption of tenants, in particular with regards to CPU, memory, and network connections. Excessive use of resources requested by a single tenant could cause runtime problems for other consumers (noisy neighbor problem). CAP helps to control resource usage:

- Business request run in isolated Java threads and hence OS thread scheduling ensures fair distribution of CPU shares. - By default, tenants have dedicated DB connection pools.

- Fine granular processing of request (CAP handlers) to avoid disproportionate blocking times of the event loop. - Tenants have dedicated DB connection pools.

::: tip Make sure that custom code doesn't introduce excessive memory or CPU consumption within a single request. ::: Because OS resources are strictly limited in a virtualized environment, a single microservice instance can handle load of a limited set of tenants, only. [**Adequate sizing**](#dos-attacks) of your microservice is mandatory, that is, adjusting memory settings, connection pool sizes, request size limits etc. according to the business needs. Last but not least you need to implement a **scaling strategy** to meet increasing load requirements by additional microservice instances. ::: warning ❗ **Sizing and scaling** is up to application developers and operators. CAP default values aren't suitable for all applications. ::: ## Secure Against Untrusted Input { #secure-untrusted-input } Without protection mechanism in place, a malicious user could misuse a valid (that is, authenticated) session with the server and attack valuable business assets. ### Injection Attacks { #injection-attacks } Attackers can send malicious input data in a regular request to make the server perform unintended actions that can lead to serious data exploits. #### Common Attack Patterns { #common-injection-attacks } - CAP's intrinsic data querying engine is immune with regards to [SQL injections](https://owasp.org/www-community/attacks/SQL_Injection) that are introduced by query parameter values that are derived from malicious user input. [CQL statements](../querying) are transformed into prepared statements that are executed in SQL databases such as SAP HANA. Be aware that injections are still possible even via CQL when the query structure (target entity, columns and so on) is based on user input:

```java String entity = ...; // from user input; String column = ...; // from user input; validate(entity, column); // for example, by comparing with positive list Select.from(entity).columns(b -> b.get(column)); ```

```js const entity = const column = validate(entity, column) // for example, by comparing with positive list SELECT.from(entity).columns(column) ```

::: warning Be careful with custom code when creating or modifying CQL queries. Additional input validation is needed when the query structure depends on the request's input. ::: - [Cross Site Scripting (XSS)](https://owasp.org/www-community/attacks/xss) is used by attackers to inject a malicious script, which is executed in the browser session of an unsuspecting user. By default, there are some protection mechanisms in place. For instance, CAP OData V4 adapter renders responses with HTTP, which prevents the browser from misinterpreting the context. On the client side, SAPUI5 provides input validation for all typed element properties and automatic output encoding in all standard controls. - Untrusted data being transferred may contain malware. [SAP Malware Scanning Service](https://help.sap.com/docs/MALWARE_SCANNING) is capable to scan provided input streams for viruses and is regularly updated. ::: warning ❗ Currently, CAP applications need to add custom handlers to **scan data being uploaded or downloaded**. ::: - [Path traversal](https://owasp.org/www-community/attacks/Path_Traversal) attacks aim to access parts of the server's file system outside the web root folder. As part of the [application zone](./overview#application-zone), an Application Router serves the static UI content of the application. The CAP microservice doesn't need to serve web content from file system. Apart from that the used web server frameworks such as Spring or Express already have adequate protection mechanisms in place. - [CLRF injections](https://owasp.org/www-community/vulnerabilities/CRLF_Injection) or [log injections](https://owasp.org/www-community/attacks/Log_Injection) can occur when untrusted user input is written to log output.

CAP Node.js offers a CLRF-safe [logging API](../../node.js/cds-log#logging-in-production) that should be used for application logs.

::: warning Currently, CAP applications need to care for escaping user data that is used as input parameter for application logging. It's recommended to make use of an existing Encoder such as OWASP [ESAPI](https://www.javadoc.io/doc/org.owasp.esapi/esapi/2.0.1/org/owasp/esapi/Encoder.html). :::

- [Deserialization of untrusted data](https://owasp.org/www-community/vulnerabilities/Deserialization_of_untrusted_data) can lead to serious exploits including remote code execution. The OData adapter converts JSON payload into an object representation. Here it follows a hardened deserialization process where the deserializer capabilities (for example, no default types in Jackson) are restricted to a minimum. A strong input validation based on EDMX model is done as well. Moreover, deserialization errors terminate the request and are tracked in the application log. #### General Recommendations Against Injections { #general-injection-attacks } In general, to achieve perfect injection resistance, applications should have input validation, output validation, and a proper Content-Security-Policy in place. - CAP provides built-in support for **input validation**. Developers can use the [`@assert`](../providing-services#input-validation) annotation to define field-specific input checks. ::: warning Applications need to validate or sanitize all input variables according to the business context. ::: - With respect to **output encoding**, CAP OData adapters have proper URI encoding for all resource locations in place. Moreover, OData validates the JSON response according to the given EDMX schema. In addition, client-side protection is given by [SAPUI5](https://pages.community.sap.com/topics/ui5) standard controls - Applications should meet basic [Content Security Policy (CSP)](https://www.w3.org/TR/CSP2/) compliance rules to further limit the attack vector on client side. CSP-compatible browsers only load resources from web locations that are listed in the allowlist defined by the server. `Content-Security-Policy` header can be set as route-specific response header in the [Application Router](https://help.sap.com/docs/btp/sap-business-technology-platform/responseheaders). SAPUI5 is [CSP-compliant](https://sapui5.hana.ondemand.com/sdk/#/topic/fe1a6dba940e479fb7c3bc753f92b28c.html) as well. ::: warning Applications have to **configure Content Security Policy** to meet basic compliance. ::: ### Service Misuse Attacks { #misues-attacks } - [Server Side Request Forgery (SSRF)](https://owasp.org/www-community/attacks/Server_Side_Request_Forgery) abuses server functionality to read or update resources from a secondary system. CAP microservices are protected from this kind of attack if they use the [CAP standard mechanisms](#authenticate-remote) for service to service communication. - [Cross-Site Request Forgery (CSRF)](https://owasp.org/www-community/attacks/csrf) attacks make end users executing unwanted actions on the server while having established a valid web session. By default, the Application Router, which manages the session with the client, enforces a CSRF token protection (on basis of `x-csrf-token` headers). Hence, CAP services don't have to deal with CSRF protection as long as they don't maintain sessions with the client. SAPUI5 supports CSRF tokens on client side out of the box. - [Clickjacking](https://owasp.org/www-community/attacks/Clickjacking) is an attack on client side where end users are tricked to open foreign pages. SAPUI5 provides [protection mechanisms](https://sapui5.hana.ondemand.com/sdk/#/topic/62d9c4d8f5ad49aa914624af9551beb7.html) against this kind of attack. ::: warning To protect SAPUI5 applications against clickjacking, configure `frame options`. ::: ### Denial-of-Service Attacks { #dos-attacks } [Denial-of-service (DoS)](https://owasp.org/www-community/attacks/Denial_of_Service) attacks attempt to reduce service availability for legitimate users. This can happen by erroneous server behavior upon a single large or a few specially crafted malicious requests that bind an excessive amount of shared OS resources such as CPU, memory, or network connections. Since OS resource allocations are distributed over the entire request, DoS-prevention needs to be addressed in all different layers of the runtime stack: #### HTTP Server and CAP Protocol Adapter The used web server frameworks such as [Spring/Tomcat](https://docs.spring.io/spring-boot/docs/current/reference/html/application-properties.html#appendix.application-properties.server) or [Express](https://expressjs.com/) start with reasonable default limits, for example: - Maximum size of the HTTP request header. - Maximum size of the HTTP request body. - Maximum queue length for incoming connection requests - Maximum number of connections that the server accepts and processes at any given time. - Connection timeout. Additional size limits and timeouts (request timeout) are established by the reverse proxy components, API Gateway and Application Router. ::: tip If you want to apply an application-specific sizing, consult the corresponding framework documentation. See section [Maximum Request Body Size](../../node.js/cds-server#maximum-request-body-size) to find out how to restrict incoming requests to a CAP Node.js application depending on the body size. ::: Moreover, CAP adapters automatically introduce query results pagination in order to limit memory peaks (customize with [`@cds.query.limit`](../providing-services#annotation-cds-query-limit)). The total number of request of OData batches can be limited by application configuration.

Settings cds.odataV4.batch.maxRequests resp. cds.odataV2.batch.maxRequests specify the corresponding limits.

::: warning ❗ CAP applications have to limit the amount of `$expands` per request in a custom handler. Also the maximum amount of requests per `$batch` request need to be configured as follows: - Node.js: cds.odata.batch_limit = \ - Java: cds.odataV4.batch.maxRequests = \ ::: ::: tip Design your CDS services exposed to web adapters on need-to-know basis. Be especially careful when exposing associations. ::: #### CAP Service Runtime Open transactions are expensive as they bind many resources such as a database connection as well as memory buffers. To minimize the amount of time a transaction must be kept open, the CAP runtime offers an [Outbox Service](../../java/outbox) that allows to schedule asynchronous remote calls in the business transaction. Hence, the request time to process a business query, which requires a remote call (such as to an audit log server or messaging broker), is minimized and independent from the response time of the remote service. ::: tip Avoid synchronous requests to remote systems during a transaction. ::: [See why CPU time is fairly distributed among business requests](#limiting-resource-consumption){.learn-more} #### Database As already outlined, database connections are a expensive resource. To limit overall usage, by default, the CAP runtime creates connection pools per subscriber tenant. Similarly, the DB driver settings such as SQL query timeout and buffer size have reasonable values. ::: tip

In case the default setting doesn't fit, connection pool properties and driver settings can be customized, respectively.

::: ::: warning ❗ Applications need to establish an adequate [Workload Management](https://help.sap.com/docs/HANA_CLOUD_DATABASE/f9c5015e72e04fffa14d7d4f7267d897/30f2e9cb92aa4f358dda4ac58e062d83.html) that controls DB resource usage. ::: #### Supplementary Measures As outlined before, a well-sized microservice instance doesn't help to protect from service downtimes when excessive workload initiated by an attacker exceeds the available capacity. [Rate limiting](https://help.sap.com/docs/btp/developing-resilient-apps-on-sap-btp/rate-limiting-c56d72711eec41118f243054f2e92f94) is a possible counter measure to restrict the frequency of calls of a client. ::: warning ❗ Applications need to establish an adequate **rate limiting** strategy. ::: There's also the possibility to introduce request filtering and rate limiting on platform level via [Route Service](https://docs.cloudfoundry.org/services/route-services.html). It has the advantage that the requests can be controlled centrally before touching application service instances. In addition, the number of instances need to be **scaled horizontally** according to current load requirements. This can be achieved automatically by consuming [Application Autoscaler](https://help.sap.com/docs/Application_Autoscaler). ### Additional Protection Mechanisms { #additional-attacks } There are additional attack vectors to consider. For instance, naive URL handling in the server endpoints frequently introduces security gaps. Luckily, CAP applications don't have to implement HTTP/URL processing on their own as CAP offers sophisticated [protocol adapters](../../about/features#consuming-services) such as OData V2/V4 that have the necessary security validations in place. The adapters also transform the HTTP requests into a corresponding CQN statement. Access control is performed on basis of CQN level according to the CDS model and hence HTTP Verb Tampering attacks are avoided. Also HTTP method override, using `X-Http-Method-Override` or `X-Http-Method` header, is not accepted by the runtime. The OData protocol allows to encode field values in query parameters of the request URL or in the response headers. This is, for example, used to specify: - [Pagination (implicit sort order)](../providing-services#pagination-sorting) - [Searching Data](../providing-services#searching-data) - Filtering ::: warning Applications need to ensure by means of CDS modeling that fields reflecting sensitive data are excluded and don't appear in URLs. ::: ::: tip It's recommended to serve all application endpoints via CAP adapters. Securing custom endpoints is left to the application. ::: In addition, CAP runs on a virtual machine with a managed heap that protects from common memory corruption vulnerabilities such as buffer overflow or range overflows. CAP also brings some tools to effectively reduce the attack vector of race condition vulnerabilities. These might be exposed when the state of resources can be manipulated concurrently and a consumer faces an unexpected state. CAP provides basic means of [concurrency control](../providing-services#concurrency-control) on different layers, for example [ETags](../providing-services#etag) and [pessimistic locks](../providing-services#select-for-update). Moreover, Messages received from the [message queue](../messaging/) are always in order. ::: tip Applications have to ensure a consistent data processing taking concurrency into account. :::

## Secure by Default and by Design { #secure-by-default } ### Secure Default Configuration { #secure-default } Where possible, CAP default configuration matches the secure by default principle: - There's no need to provide any password, credentials, or certificates to [protect communication](#secure-communications). - A CAP application bound to an XSUAA instance authenticates all endpoints [by default](#secure-authentication). Developers have to explicitly configure public endpoints if necessary. - Isolated multitenancy is provided out of the box. - Application logging has `INFO` level to avoid potential information disclosures. - CAP also has first-class citizen support for [Fiori UI](../../advanced/fiori) framework that brings a lot secure by default features in the UI client. Of course, several security aspects need application-specific configuration. For instance, this is true for [authorizations](#secure-authorization) or application [sizing](#dos-attacks). ::: tip It's recommended to ensure security settings by automated integration tests. ::: ### Fail Securely { #fail-securely } CAP runtime differentiates several types of error situations during request processing: - Exceptions because of invalid user input (HTTP 4xx). - Exceptions because of unexpected server behaviour, for example, network issues. - Unrecoverable errors due to serious issues in the VM (for example, lack of memory) or program flaws. In general, **exceptions immediately stop the execution of the current request**. In Java, the thrown [ServiceException](https://www.javadoc.io/doc/com.sap.cds/cds-services-api/latest/com/sap/cds/services/EventContext.html) is automatically scoped to the current request by means of thread isolation. { .java } CAP Node.js adds an exception wrapper to ensure that only the failing request is affected by the exception. { .node } Customers can react in dedicated exception handlers if necessary. In contrast, **errors stop the overall microservice** to ensure that security measures aren't weakened. ::: tip Align the exception handling in your custom coding with the provided exception handling capabilities of the CAP runtime. :::

# Data Protection & Privacy This section describes how you can make your CAP application compliant with data protection and privacy requirements. ## General Statement { #dpp-statement } Governments place legal requirements on industry to protect data and privacy. ::: tip No guide, including this one, concerning CAP attempts to give any advice on whether any features and functions are the best method to support company-, industry-, region-, or country-specific requirements. Furthermore, the information provided in this note does not give any advice or recommendations with regards to additional features that might be required in a particular environment. Decisions related to data protection must be made on a case-by-case basis, under consideration of the given system landscape and the applicable legal requirements. ::: For general information about data protection and privacy (DPP) on SAP BTP, see the SAP BTP documentation under [Data Protection and Privacy](https://help.sap.com/docs/btp/sap-business-technology-platform/data-protection-and-privacy). ## Data Protection & Privacy in CAP { #dpp-cap } CAP is a framework that provides modeling and runtime features to enable customers to build business applications on top. As a framework, in general, CAP doesn't store or manage any personal data on its own with some exceptions: - Application logging on detailed level written by CAP runtime might contain personal data such as user names and IP addresses. The logs are mandatory to operate the system. Connect an adequate logging service to meet compliance requirements such as [SAP Application Logging Service](https://help.sap.com/docs/application-logging-service/sap-application-logging-service/sap-application-logging-service-for-cloud-foundry-environment). - A draft-enabled service `Foo` has an entity `Foo.DraftAdministrativeData` with fields `CreatedByUser`, `InProcessByUser` and `LastChangedByUser` containing personal data for all draft entity instances in edit mode. - Messages temporarily written to transaction outbox might contain personal data. The entries are mandatory to operate the system. If necessary, applications can process these messages by standard CAP functionality (CDS model `@sap/cds/srv/outbox`). - Be aware that personal data might be added automatically when using the [managed](../domain-modeling#managed-data) aspect. Dependent on the business scenario, custom CDS models served by CAP runtime will most likely contain personal data that is also stored in a backing service. CAP provides a [rich set of tools](aspects) to protect the application from unauthorized access to business data, including personal data. Furthermore, it helps applications to provide [higher-level DPP-related functions](#dpp-support) such as data retrieval. ::: warning ❗ **Applications are responsible to implement compliance requirements with regards to data protection and privacy according to their specific use case**. ::: Also refer to related guides of most important platform services: [SAP Cloud Identity Services - Configuring Privacy Policies](https://help.sap.com/docs/IDENTITY_AUTHENTICATION/6d6d63354d1242d185ab4830fc04feb1/ed48466d770f4519aa23bba754851fbd.html){.learn-more} [SAP HANA Cloud - Data Protection and Privacy](https://help.sap.com/docs/HANA_CLOUD_DATABASE/c82f8d6a84c147f8b78bf6416dae7290/ad9588189e844092910103f2f7b1c968.html){.learn-more} ## Data Protection & Privacy Supported by CAP { #dpp-support } CAP provides several [features](../data-privacy/) to help applications meet DPP-requirements: - The [Personal Data Management (PDM)](../data-privacy/pdm) integration has a configurable **retrieval function**, which can be used to inform data subjects about personal data stored related to them. - CAP also provides a *fully model-driven* approach to track **changes in personal data** or **read access to sensitive personal data** in the audit log. Having [declared personal data](../data-privacy/annotations) in your model, CAP automatically triggers corresponding [audit log events](../data-privacy/audit-logging). ::: warning ❗ So far, applications have to integrate [SAP Data Retention Manager](https://help.sap.com/docs/DATA_RETENTION_MANAGER) to implement an adequate **erasure function** for personal data out of retention period. CAP will cover an out-of-the-box integration in the future. ::: # Managing Data Privacy CAP helps application projects to comply with data privacy regulations using SAP Business Technology Platform (BTP) services. Find a step-by-step guide to these hereinafter... ::: warning SAP does not give any advice on whether the features and functions provided to facilitate meeting data privacy obligations are the best method to support company, industry, regional, or country/region-specific requirements. Furthermore, this information should not be taken as advice or a recommendation regarding additional features that would be required in specific IT environments. Decisions related to data protection must be made on a case-by-case basis, considering the given system landscape and the applicable legal requirements. ::: ## Introduction to Data Privacy Data protection is associated with numerous legal requirements and privacy concerns, such as the EU's [General Data Protection Regulation](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation). In addition to compliance with general data protection and privacy acts regarding [personal data](https://en.wikipedia.org/wiki/Personal_data), you need to consider compliance with industry-specific legislation in different countries/regions. CAP supports applications in their obligations to comply to data privacy regulations, by automating tedious tasks as much as possible based on annotated models. That is, CAP provides easy ways to designate personal data, as well as out-of-the-box integration with SAP BTP services, which enable you to fulfill specific data privacy requirements in your application. This relieves application developers of these tedious tasks and related efforts. ![Shows with which solutions CAP annotations can be used out of the box, as described in the following table.](./assets/Data-Privacy.drawio.svg){} ### In a Nutshell The most essential requests you have to answer are those in the following table. The table also shows the basis of the requirement and the corresponding discipline for the request: | Question / Request | Obligation | Solution | | ------------------------------------------- | ----------------------------------------------- | ----------------------------------- | | *What data about me do you have stored?* | [Right of access](#right-of-access) | [Personal Data Management](pdm.md) | | *Please delete all personal data about me!* | [Right to be forgotten](#right-to-be-forgotten) | [Data Retention Management](drm.md) | | *When was personal data stored/changed?* | [Transparency](#transparency) | [Audit Logging](audit-logging.md) | ## Annotating Personal Data The first and frequently only task to do as an application developer is to identify entities and elements (potentially) holding personal data using `@PersonalData` annotations. These are used to automate CAP-facilitated audit logging, personal data management, and data retention management as much as possible. [Learn more in the *Annotating Personal Data* chapter](annotations) {.learn-more} ## Automatic Audit Logging {#transparency} The **Transparancy** obligation, requests to be able to report with whom data stored about an individual is shared and where that came from (for example, [EU GDPR Article 15(1)(c,g)](https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:02016R0679-20160504&qid=1692819634946#tocId22)). The [SAP Audit Log Service](https://help.sap.com/docs/btp/sap-business-technology-platform/audit-logging-in-cloud-foundry-environment) stores all audit logs for a tenant in a common, compliant data store and allows auditors to search through and retrieve the respective logs when necessary. [Learn more in the *Audit Logging* guide](audit-logging) {.learn-more} ## Personal Data Management { #right-of-access } The [**Right of Access** to personal data](https://en.wikipedia.org/wiki/Right_of_access_to_personal_data) "gives people the right to access their personal data and information about how this personal data is being processed". The [SAP Personal Data Manager](https://help.sap.com/docs/personal-data-manager) allows you to inform individuals about the data you have stored regarding them. [Learn more in the *Personal Data Management* guide](pdm) {.learn-more} ## Data Retention Management { #right-to-be-forgotten } The [**Right to be Forgotten**](https://en.wikipedia.org/wiki/Right_to_be_forgotten) gives people "the right to request erasure of personal data related to them on any one of a number of grounds [...]". The [SAP Data Retention Manager](https://help.sap.com/docs/data-retention-manager) allows you to manage retention and residence rules to block or destroy personal data. # Annotating Personal Data In order to automate audit logging, personal data management, and data retention management as much as possible, the first and frequently only task to do as an application developer is to identify entities and elements (potentially) holding personal data using `@PersonalData` annotations. ## Reference App Sample { #annotated-model } In the remainder of this guide, we use the [Incidents Management reference sample app](https://github.com/cap-js/incidents-app) as the base to add data privacy and audit logging to. ![Shows the connections between the entities in the sample app.](./assets/Incidents-App.drawio.svg){} So, let's annotate the data model to identify personal data. In essence, in all our entities we search for elements which carry personal data, such as person names, birth dates, etc., and tag them accordingly. All found entities are classified as either *Data Subjects*, *Subject Details* or *Related Data Objects*. Following the [best practice of separation of concerns](../domain-modeling#separation-of-concerns), we annotate our domain model in a separate file *srv/data-privacy.cds*, which we add to our project and fill it with the following content: > For the time beeing also replace the data in _data/sap.capire.incidents-Customers.csv_. ::: code-group ```cds [db/data-privacy.cds] using { sap.capire.incidents as my } from '../db/schema'; extend my.Customers with { dateOfBirth : Date; }; annotate my.Customers with @PersonalData : { DataSubjectRole : 'Customer', EntitySemantics : 'DataSubject' } { ID @PersonalData.FieldSemantics: 'DataSubjectID'; firstName @PersonalData.IsPotentiallyPersonal; lastName @PersonalData.IsPotentiallyPersonal; email @PersonalData.IsPotentiallyPersonal; phone @PersonalData.IsPotentiallyPersonal; dateOfBirth @PersonalData.IsPotentiallyPersonal; creditCardNo @PersonalData.IsPotentiallySensitive; }; annotate my.Addresses with @PersonalData: { EntitySemantics : 'DataSubjectDetails' } { customer @PersonalData.FieldSemantics: 'DataSubjectID'; city @PersonalData.IsPotentiallyPersonal; postCode @PersonalData.IsPotentiallyPersonal; streetAddress @PersonalData.IsPotentiallyPersonal; }; annotate my.Incidents with @PersonalData : { EntitySemantics : 'Other' } { customer @PersonalData.FieldSemantics: 'DataSubjectID'; }; ``` ```csv [data/sap.capire.incidents-Customers.csv] ID,firstName,lastName,email,phone,dateOfBirth 1004155,Daniel,Watts,daniel.watts@demo.com,+44-555-123,1996-01-01 1004161,Stormy,Weathers,stormy.weathers@demo.com,,1981-01-01 1004100,Sunny,Sunshine,sunny.sunshine@demo.com,+01-555-789,1965-01-01 ``` ::: ## @PersonalData... Let's break down the annotations to identify personal data, shown in the sample above. These annotations fall into three categories: - **Entity-level annotations** signify relevant entities as *Data Subjects*, *Data Subject Details*, or *Related Data Objects* in data privacy terms, as depicted in the graphic below. - **Key-level annotations** signify object primary keys, as well as references to data subjects (which have to be present on each object). - **Field-level annotations** identify elements containing personal data. Learn more about these annotations in the [@PersonalData OData vocabulary](https://github.com/SAP/odata-vocabularies/blob/main/vocabularies/PersonalData.md). {.learn-more} ### .EntitySemantics {.annotation} The entity-level annotation `@PersonalData.EntitySemantics` signifies relevant entities as *Data Subject*, *Data Subject Details*, or *Other* in data privacy terms, as depicted in the following graphic. ![Shows the connections between the entities in the sample app. In addition via color coding it makes clear how entities are annotated: customers are data subject, addresses are data subject details and incidents are other.](./assets/Data-Subjects.drawio.svg){} The following table provides some further details. Annotation | Description --------------------- | ------------- `DataSubject` | The entities of this set describe a data subject (an identified or identifiable natural person), for example, Customer or Vendor. `DataSubjectDetails` | The entities of this set contain details of a data subject (an identified or identifiable natural person) but do not by themselves identify/describe a data subject, for example, Addresses. `Other` | Entities containing personal data or references to data subjects, but not representing data subjects or data subject details by themselves. For example, customer quote, customer order, or purchase order with involved business partners. These entities are relevant for audit logging. There are no restrictions on their structure. The properties should be annotated suitably with `FieldSemantics`. Hence, we annotate our model as follows: ```cds annotate my.Customers with @PersonalData: { EntitySemantics: 'DataSubject' // [!code focus] }; annotate my.Addresses with @PersonalData: { EntitySemantics: 'DataSubjectDetails' // [!code focus] }; annotate my.Incidents with @PersonalData: { EntitySemantics: 'Other' // [!code focus] }; ``` ### .DataSubjectRole {.annotation} Can be added to `@PersonalData.EntitySemantics: 'DataSubject'`. It's a user-chosen string specifying the role name to use. If omitted, the default is the entity name. Use case is similar to providing user-friendly labels for the UI, although in this case there's no i18n. In our model, we can add the `DataSubjectRole` as follows: ```cds annotate my.Customers with @PersonalData: { EntitySemantics: 'DataSubject', DataSubjectRole: 'Customer' // [!code focus] }; ``` ### .FieldSemantics: DataSubjectID {.annotation} Use this annotation to identify data subject's unique key, or a reference to it. References are commonly associations or foreign keys in subject details entities, or related ones, referring to a subject entity. - Each `@PersonalData` entity needs to identify a the `DataSubjectID` element. - For entities with `DataSubject` semantics, this is typically the primary key. - For entities with `DataSubjectDetails` or `Other` semantics, this is usually an association to the data subject. Hence, we annotate our model as follows: ```cds annotate my.Customers with { ID @PersonalData.FieldSemantics: 'DataSubjectID' // [!code focus] }; annotate my.Addresses with { customer @PersonalData.FieldSemantics: 'DataSubjectID' // [!code focus] }; annotate my.Incidents with { customer @PersonalData.FieldSemantics: 'DataSubjectID' // [!code focus] }; ``` ### .IsPotentiallyPersonal {.annotation} `@PersonalData.IsPotentiallyPersonal` tags which fields are personal and, for example, require audit logs if modified. ```cds annotate my.Customers with { firstName @PersonalData.IsPotentiallyPersonal; // [!code focus] lastName @PersonalData.IsPotentiallyPersonal; // [!code focus] email @PersonalData.IsPotentiallyPersonal; // [!code focus] phone @PersonalData.IsPotentiallyPersonal; // [!code focus] }; ``` ### .IsPotentiallySensitive {.annotation} `@PersonalData.IsPotentiallySensitive` tags which fields are sensitive and, for example, require audit logs in case of access. ```cds annotate my.Customers with { creditCardNo @PersonalData.IsPotentiallySensitive; // [!code focus] }; ``` ## Next Steps... Having annotated your data model with `@PersonalData` annotations, you can now go on to the respective tasks that leverage these annotations to automate as much as possible: - [*Automated Audit Logging*](audit-logging) - [*Personal Data Management*](pdm) - [*Data Retention Management*](drm) # Audit Logging The [`@cap-js/audit-logging`](https://www.npmjs.com/package/@cap-js/audit-logging) plugin provides out-of-the box support for automatic audit logging of data privacy-related events, in particular changes to *personal data* and reads of *sensitive* data. Find here a step-by-step guide how to use it. :::warning _The following is mainly written from a Node.js perspective. For Java's perspective, please see [Java - Audit Logging](../../java/auditlog)._ ::: ## Annotate Personal Data First identify entities and elements (potentially) holding personal data using `@PersonalData` annotations, as explained in detail in the [*Annotating Personal Data* chapter](annotations) of these guides. > We keep using the [Incidents Management reference sample app](https://github.com/cap-js/incidents-app). ## Add the Plugin { #setup } To enable automatic audit logging simply add the [`@cap-js/audit-logging`](https://www.npmjs.com/package/@cap-js/audit-logging) plugin package to your project like so: ```sh npm add @cap-js/audit-logging ``` ::: details Behind the Scenes… [CDS Plugin Packages](../../node.js/cds-plugins) are self-contained extensions. They not only include the relevant code but also bring their own default configuration. In our case, next to bringing the respective code, the plugin does the following: 1. Sets cds.requires.audit-log: true 2. Which in turn activates the effective `audit-log` configuration via **presets**: ```jsonc { "audit-log": { "handle": ["READ", "WRITE"], "outbox": true, "[development]": { "kind": "audit-log-to-console" }, "[hybrid]": { "kind": "audit-log-to-restv2" }, "[production]": { "kind": "audit-log-to-restv2" } }, "kinds": { "audit-log-to-console": { "impl": "@cap-js/audit-logging/srv/log2console" }, "audit-log-to-restv2": { "impl": "@cap-js/audit-logging/srv/log2restv2", "vcap": { "label": "auditlog" } } } } ``` **The individual configuration options are:** - `impl` — the service implementation to use - `outbox` — whether to use transactional outbox or not - `handle` — which events (`READ` and/or `WRITE`) to intercept and generate log messages from **The preset uses profile-specific configurations** for (hybrid) development and production. Use the `cds env` command to find out the effective configuration for your current environment: ::: code-group ```sh [w/o profile] cds env requires.audit-log ``` ```sh [production profile] cds env requires.audit-log --profile production ``` ::: ## Test-drive Locally The previous step is all we need to do to automatically log personal data-related events. Let's see that in action… 1. **Start the server** as usual: ```sh cds watch ``` 2. **Send an update** request that changes personal data: ::: code-group ```http [test/audit-logging.http] PATCH http://localhost:4004/admin/Customers(2b87f6ca-28a2-41d6-8c69-ccf16aa6389d) HTTP/1.1 Authorization: Basic alice:in-wonderland Content-Type: application/json { "firstName": "Jane", "lastName": "Doe" } ``` ::: [Find more sample requests in the Incident Management sample.](https://github.com/cap-js/incidents-app/blob/attachments/test/audit-logging.http){.learn-more} 3. **See the audit logs** in the server's console output: ```js { data_subject: { id: { ID: '2b87f6ca-28a2-41d6-8c69-ccf16aa6389d' }, role: 'Customer', type: 'AdminService.Customers' }, object: { type: 'AdminService.Customers', id: { ID: '2b87f6ca-28a2-41d6-8c69-ccf16aa6389d' } }, attributes: [ { name: 'firstName', old: 'Sunny', new: 'Jane' }, { name: 'lastName', old: 'Sunshine', new: 'Doe' } ], uuid: '5cddbc91-8edf-4ba2-989b-87869d94070d', tenant: 't1', user: 'alice', time: 2024-02-08T09:21:45.021Z } ``` ## Use SAP Audit Log Service While we simply dumped audit log messages to stdout in local development, we'll be using the SAP Audit Log Service on SAP BTP in production. Following is a brief description of the necessary steps for setting this up. A more comprehensive guide, incl. tutorials, is currently under development. ### Setup Instance and Deploy App For deployment in general, please follow the [deployment guide](../deployment/). Check the rest of this guide before actually triggering the deployment (that is, executing `cf deploy`). Here is what you need to do additionally, to integrate with SAP Audit Log Service: 1. In your space, create a service instance of the _SAP Audit Log Service_ (`auditlog`) service with plan `premium`. 2. Add the service instance as _existing resource_ to your `mta.yml` and bind it to your application in its _requires_ section. Existing resources are defined like this: ```yml resources: - name: my-auditlog-service type: org.cloudfoundry.existing-service ``` [Learn more about *Audit Log Write API for Customers*](https://help.sap.com/docs/btp/sap-business-technology-platform/audit-log-write-api-for-customers?version=Cloud){.learn-more} ### Accessing Audit Logs There are two options to access audit logs: 1. Create an instance of service `auditlog-management` to retrieve audit logs via REST API, see [Audit Log Retrieval API Usage for the Cloud Foundry Environment](https://help.sap.com/docs/btp/sap-business-technology-platform/audit-log-retrieval-api-usage-for-subaccounts-in-cloud-foundry-environment). 2. Use the SAP Audit Log Viewer, see [Audit Log Viewer for the Cloud Foundry Environment](https://help.sap.com/docs/btp/sap-business-technology-platform/audit-log-viewer-for-cloud-foundry-environment). ## Generic Audit Logging ### Behind the Scenes... For all [defined services](../providing-services#service-definitions), the generic audit logging implementation does the following: - Intercept all write operations potentially involving personal data. - Intercept all read operations potentially involving sensitive data. - Determine the affected fields containing personal data, if any. - Construct log messages, and send them to the connected audit log service. - All emitted log messages are sent through the [transactional outbox](#transactional-outbox). - Apply resiliency mechanisms like retry with exponential backoff, and more. ## Custom Audit Logging { #custom-audit-logging } In addition to the generic audit logging provided out of the box, applications can also log custom events with custom data using the programmatic API. Connecting to the service: ```js const audit = await cds.connect.to('audit-log') ``` Sending log messages: ```js await audit.log('Foo', { bar: 'baz' }) ``` ::: tip Audit Logging as Just Another CAP Service The Audit Log Service API is implemented as a CAP service, with the service API defined in CDS as shown in the next section. In effect, the common patterns of [*CAP Service Consumption*](../using-services) apply, as well as all the usual benefits like *mocking*, *late-cut µ services*, *resilience* and *extensibility*. ::: ### Service Definition Below is the complete reference modeling as contained in `@cap-js/audit-logging`. The individual operations and events are briefly discussed in the following sections. The service definition declares the generic `log` operation, which is used for all kinds of events, as well as the common type `LogEntry`, which declares the common fields of all log messages. These fields are filled in automatically by the base service and any values provided by the caller are ignored. Further, the service has pre-defined event payloads for the four event types: 1. _Log read access to sensitive personal data_ 1. _Log changes to personal data_ 1. _Security event log_ 1. _Configuration change log_ These payloads are based on [SAP Audit Log Service's REST API](https://help.sap.com/docs/btp/sap-business-technology-platform/audit-log-write-api-for-customers), which maximizes performance by omitting any intermediate data structures. ```cds namespace sap.auditlog; service AuditLogService { action log(event : String, data : LogEntry); event SensitiveDataRead : LogEntry { data_subject : DataSubject; object : DataObject; attributes : many { name : String; }; attachments : many { id : String; name : String; }; channel : String; }; event PersonalDataModified : LogEntry { data_subject : DataSubject; object : DataObject; attributes : many Modification; success : Boolean default true; }; event ConfigurationModified : LogEntry { object : DataObject; attributes : many Modification; }; event SecurityEvent : LogEntry { data : {}; ip : String; }; } /** Common fields, filled in automatically */ type LogEntry { uuid : UUID; tenant : String; user : String; time : Timestamp; } type DataObject { type : String; id : {}; } type DataSubject : DataObject { role : String; } type Modification { name : String; old : String; new : String; } ``` ### Sensitive Data Read ```cds event SensitiveDataRead : LogEntry { data_subject : DataSubject; object : DataObject; attributes : many { name : String; }; attachments : many { id : String; name : String; }; channel : String; } type DataObject { type : String; id : {}; } type DataSubject : DataObject { role : String; } ``` Send `SensitiveDataRead` event log messages like that: ```js await audit.log ('SensitiveDataRead', { data_subject: { type: 'sap.capire.bookshop.Customers', id: { ID: '1923bd11-b1d6-47b6-a91b-732e755fa976' }, role: 'Customer', }, object: { type: 'sap.capire.bookshop.BillingData', id: { ID: '399a2704-3d2d-4fa1-9e7d-a4e45c67749b' } }, attributes: [ { name: 'creditCardNo' } ] }) ``` ### Personal Data Modified ```cds event PersonalDataModified : LogEntry { data_subject : DataSubject; object : DataObject; attributes : many Modification; success : Boolean default true; } type Modification { name : String; old : String; new : String; } ``` Send `PersonalDataModified` event log messages like that: ```js await audit.log ('PersonalDataModified', { data_subject: { type: 'sap.capire.bookshop.Customers', id: { ID: '1923bd11-b1d6-47b6-a91b-732e755fa976' }, role: 'Customer', }, object: { type: 'sap.capire.bookshop.Customers', id: { ID: '1923bd11-b1d6-47b6-a91b-732e755fa976' } }, attributes: [ { name: 'emailAddress', old: 'foo@example.com', new: 'bar@example.com' } ] }) ``` ### Configuration Modified ```cds event ConfigurationModified : LogEntry { object : DataObject; attributes : many Modification; } ``` Send `ConfigurationModified` event log messages like that: ```js await audit.log ('ConfigurationModified', { object: { type: 'sap.common.Currencies', id: { ID: 'f79ba248-c348-4962-9fef-680c3b88807c' } }, attributes: [ { name: 'symbol', old: 'EUR', new: '€' } ] }) ``` ### Security Events ```cds event SecurityEvent : LogEntry { data : {}; ip : String; } ``` Send `SecurityEvent` log messages like that: ```js await audit.log ('SecurityEvent', { data: { user: 'alice', action: 'Attempt to access restricted service "PDMService" with insufficient authority' }, ip: '127.0.0.1' }) ``` > In the SAP Audit Log Service REST API, `data` is a String. For ease of use, the default implementation stringifies `data`, if it is provided as an object. [Custom implementations](#custom-implementation) should also handle both. ## Custom Implementation { #custom-implementation } In addition, everybody could provide new implementations in the same way as we implement the mock variant: ```js const { AuditLogService } = require('@cap-js/audit-logging') class MyAuditLogService extends AuditLogService { async init() { this.on('*', function (req) { // [!code focus] const { event, data } = req console.log(`[my-audit-log] - ${event}:`, data) }) return super.init() } } module.exports = MyAuditLogService ``` As always, custom implementations need to be configured in `cds.requires.<>.impl`: ```json { "cds": { "requires": { "audit-log": { "impl": "lib/MyAuditLogService.js" } } } } ``` ## Transactional Outbox { #transactional-outbox } By default, all log messages are sent through a transactional outbox. This means, when sent, log messages are first stored in a local outbox table, which acts like a queue for outbound messages. Only when requests are fully and successfully processed, these messages are forwarded to the audit log service. ![This graphic is explained in the accompanying text.](./assets/Transactional-Outbox.drawio.svg) This provides an ultimate level of resiliency, plus additional benefits: - **Audit log messages are guaranteed to be delivered** — even if the audit log service should be down for a longer time period. - **Asynchronous delivery of log messages** — the main thread doesn't wait for requests being sent and successfully processed by the audit log service. - **False log messages are avoided** — messages are forwarded to the audit log service on successfully committed requests; and skipped in case of rollbacks. This transparently applies to all implementations, even [custom implementations](#custom-implementation). You can opt out of this default by configuring cds.audit-log.[development].outbox = false. # Personal Data Management Use the SAP Personal Data Manager (PDM) with a CAP application. :::warning To follow this cookbook hands-on you need an enterprise account. The SAP Personal Data Manager service is currently only available for [enterprise accounts](https://discovery-center.cloud.sap/missiondetail/3019/3297/). An entitlement in trial accounts is not possible. ::: SAP BTP provides the [*SAP Personal Data Manager (PDM)*](https://help.sap.com/docs/PERSONAL_DATA_MANAGER) which allows administrators to respond to the question "What data of me do you have?". To answer this question, the PDM service needs to fetch all personal data using an OData endpoint. That endpoint has to be provided by the application as follows. ## Annotate Personal Data First identify entities and elements (potentially) holding personal data using `@PersonalData` annotations, as explained in detail in the [*Annotating Personal Data* chapter](annotations) of these guides. > We keep using the [Incidents Management reference sample app](https://github.com/cap-js/incidents-app). ## Provide a Service Interface to SAP Personal Data Manager SAP Personal Data Manager needs to call into your application to read personal data so you have to define a respective service endpoint, complying to the interface required by SAP Personal Data Manager. Following the CAP principles, we recommend adding a new dedicated CAP service that handles all the personal data manager requirements for you. This keeps the rest of your data model clean and enables reuse, just as CAP promotes it. ### CAP Service Model for SAP Personal Data Manager Following the [best practice of separation of concerns](../domain-modeling#separation-of-concerns), we create a dedicated service for the integration with SAP Personal Data Manager: ::: code-group ```cds [srv/pdm-service.cds] using {sap.capire.incidents as db} from '../db/schema'; @requires: 'PersonalDataManagerUser' // security check service PDMService @(path: '/pdm') { // Data Privacy annotations on 'Customers' and 'Addresses' are derived from original entity definitions entity Customers as projection on db.Customers; entity Addresses as projection on db.Addresses; entity Incidents as projection on db.Incidents // create view on Incidents and Conversations as flat projection entity IncidentConversationView as select from Incidents { ID, title, urgency, status, key conversation.ID as conversation_ID, conversation.timestamp as conversation_timestamp, conversation.author as conversation_author, conversation.message as conversation_message, customer.ID as customer_ID, customer.email as customer_email }; // annotate new view annotate PDMService.IncidentConversationView with @(PersonalData.EntitySemantics: 'Other') { customer_ID @PersonalData.FieldSemantics: 'DataSubjectID'; }; // annotations for Personal Data Manager - Search Fields annotate Customers with @(Communication.Contact: { n : { surname: lastName, given : firstName }, bday : dateOfBirth, email: [{ type : #preferred, address: email}] }); }; ``` ::: ::: tip Make sure to have [indicated all relevant entities and elements in your domain model](annotations). ::: ### Provide Flat Projections As an additional step, you have to create flat projections on the additional business data, like transactional data. In our model, we have `Incidents` and `Conversations`, which are connected via a [composition](https://github.com/SAP-samples/cloud-cap-samples/blob/gdpr/orders/db/schema.cds). Since SAP Personal Data Manager needs flattened out structures, we define a helper view `IncidentConversationView` to flatten this out. We have to then add data privacy-specific annotations to this new view as well. The `IncidentConversationView` as transactional data is marked as `Other`. In addition, it is important to tag the correct field, which defines the corresponding data subject, in our case that is `customer_ID @PersonalData.FieldSemantics: 'DataSubjectID';` ### Annotating Search Fields In addition, the most important search fields of the data subject have to be annotated with the corresponding annotation `@Communication.Contact`. To perform a valid search in the SAP Personal Data Manager application, you will need _Surname_, _Given Name_, and _Email_ or the _Data Subject ID_. Details about this annotation can be found in [Communication Vocabulary](https://github.com/SAP/odata-vocabularies/blob/main/vocabularies/Communication.md). Alternatively to the tuple _Surname_, _Given Name_, and _Email_, you can also use _Surname_, _Given Name_, and _Birthday_ (called `bday`), if available in your data model. Details about this can be found in [SAP Personal Data Manager - Developer Guide](https://help.sap.com/docs/personal-data-manager/4adcd96ce00c4f1ba29ed11f646a5944/v4-annotations?q=Contact&locale=en-US). ### Restrict Access Using the `@requires` Annotation To restrict access to this sensitive data, the `PDMservice` is protected by the `@requires: 'PersonalDataManagerUser'` annotation. Calling the `PDMservice` externally without the corresponding permission is forbidden. The Personal Data Manager service calls the `PDMservice` with the needed role granted. This is configured in the _xs-security.json_ file, which is explained later. [Learn more about security configuration and the SAP Personal Data Manager.](https://help.sap.com/docs/PERSONAL_DATA_MANAGER/620a3ea6aaf64610accdd05cca9e3de2/4ee5705b8ded43e68bde610223722971.html#loio8eb6d9f889594a2d98f478bd57412ceb){.learn-more} At this point, you are done with your application. Let's set up the SAP Personal Data Manager and try it out. ## Connecting SAP Personal Data Manager Next, we will briefly detail the integration to SAP Personal Data Manager. A more comprehensive guide, incl. tutorials, is currently under development. For further details, see the [SAP Personal Data Manager Developer Guide](https://help.sap.com/docs/personal-data-manager/4adcd96ce00c4f1ba29ed11f646a5944/what-is-personal-data-manager). ### Activate Access Checks in _xs-security.json_ Because we protected the `PDMservice`, we need to establish the security check properly. In particular, you need the _xs-security.json_ file to make the security check active. The following _xs-security.json_ is from our sample. ```json { "xsappname": "incidents-mgmt", "tenant-mode": "shared", "scopes": [ { "name": "$XSAPPNAME.PersonalDataManagerUser", "description": "Authority for Personal Data Manager", "grant-as-authority-to-apps": [ "$XSSERVICENAME(pdm)" ] } ] } ``` Here you define that your personal data manager service instance, called `pdm`, is allowed to access your CAP application granting the `PersonalDataManagerUser` role. ### Add `@sap/xssec` Library To make the authentication work, you have to enable the security strategy by installing the `@sap/xssec` package: ```sh npm install @sap/xssec ``` [Learn more about authorization in CAP using Node.js.](../../node.js/authentication#jwt){.learn-more} ### Build and Deploy Your Application The Personal Data Manager can't connect to your application running locally. Therefore, you first need to deploy your application. In our sample, we added two manifest files using `cds add cf-manifest` and SAP HANA configuration using `cds add hana`. The general deployment is described in detail in [Deploy Using Manifest Files](../deployment/to-cf). Make a production build: ```sh cds build --production ``` Deploy your application: ```sh cf create-service-push ``` ### Subscribe to SAP Personal Data Manager Service [Subscribe to the service](https://help.sap.com/docs/PERSONAL_DATA_MANAGER/620a3ea6aaf64610accdd05cca9e3de2/ef10215655a540b6ba1c02a96e118d66.html) from the _Service Marketplace_ in the SAP BTP cockpit. ![A screenshot of the tile in the cockpit for the SAP Personal Data Manager service.](assets/pdmCockpitCreate.png){} Follow the wizard to create your subscription. ### Create Role Collections SAP Personal Data Manager comes with the following roles: Role Name | Role Template ----------|------ PDM_Administrator | PDM_Administrator PDM_CustomerServiceRepresentative | PDM_CustomerServiceRepresentative PDM_OperatorsClerk | PDM_OperatorsClerk All of these roles have two different _Application Identifiers_. ::: tip Application identifiers with **!b** are needed for the UI, and identifiers with **!t** are needed for executing the Postman collection. ::: [Learn more about defining a role collection in SAP BTP cockpit](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/4b20383efab341f181becf0a947a5498.html){.learn-more} ### Create a Service Instance You need a configuration file, like the following, to create a service instance for the Personal Data Manager. `pdm-instance-config.json` ```json { "xs-security": { "xsappname": "incidents-mgmt", "authorities": ["$ACCEPT_GRANTED_AUTHORITIES"] }, "fullyQualifiedApplicationName": "incidents-mgmt", "appConsentServiceEnabled": true } ``` Create a service instance using the SAP BTP cockpit or execute the following command: ```sh cf create-service personal-data-manager-service standard incidents-mgmt-pdm -c ./pdm-instance-config.json ``` ### Bind the Service Instance to Your Application. With both the application deployed and the SAP Personal Data Manger service set up, you can now bind the service instance of the Personal Data Manager to your application. Use the URL of your application in a configuration file, such as the following example, which you need when binding a service instance. `pdm-binding-config.json` ```json { "fullyQualifiedApplicationName": "incidents-mgmt", "fullyQualifiedModuleName": "incidents-mgmt-srv", "applicationTitle": "PDM Incidents", "applicationTitleKey": "PDM Incidents", "applicationURL": "https://incidents-mgmt-srv.cfapps.eu10.hana.ondemand.com/", // get the URL from the CF CLI command: cf apps "endPoints": [ { "type": "odatav4", "serviceName": "pdm-service", "serviceTitle": "Incidents Management", "serviceTitleKey": "IncidentsManagement", "serviceURI": "pdm", "hasGdprV4Annotations": true, "cacheControl": "no-cache" } ] } ``` Here the `applicationURL`, the `fullyQualifiedModuleName`, and the `serviceURI` have to be those of your Cloud Foundry deployment and your CAP service definition (_services-manifest.yaml_). Bind the service instance using the SAP BTP cockpit or execute the following command: ```sh cf bind-service incidents-mgmt-srv incidents-mgmt-pdm -c ./pdm-binding-config.json ``` ## Using the SAP Personal Data Manager Application Open the SAP Personal Data Manager application from the _Instances and Subscriptions_ page in the SAP BTP cockpit. ![To open the application, open the three dot menu and select "Go to Application".](assets/pdmCockpit.png){} In the personal data manager application you can search for data subjects with _First Name_, _Last Name_, and _Date of Birth_, or alternatively with their _ID_. ![A screenshot of the SAP Personal Data Manager application.](assets/pdmApplication.png){} # Deployment Learn here about deployment options of a CAP application. # Deploy to Cloud Foundry A comprehensive guide on deploying applications built with SAP Cloud Application Programming Model (CAP) to SAP BTP Cloud Foundry environment. ## Intro & Overview After completing the functional implementation of your CAP application by following the [Getting Started](../../get-started/in-a-nutshell) or [Cookbook](../) guides, you would finally deploy it to the cloud for production. The essential steps are illustrated in the following graphic: ![First prepare for production (once) and then freeze your dependencies (once and on upgrades). Next build and assemble and then deploy.](assets/deploy-setps.drawio.svg){} First, you apply these steps manually in an ad-hoc deployment, as described in this guide. Then, after successful deployment, you automate them using [CI/CD pipelines](cicd). ## Prerequisites The following sections are based on a new project that you can create like this:

```sh cds init bookshop --add sample cd bookshop ``` ::: details Alternatively, download or clone the sample repository Exercise the following steps in the `bookshop` subfolder of the [`cloud-cap-samples`](https://github.com/sap-samples/cloud-cap-samples) repo: ```sh git clone https://github.com/sap-samples/cloud-cap-samples samples cd samples/bookshop ``` :::

```sh cds init bookshop --java --add sample cd bookshop ``` > If you want to use a ready-to-be-deployed sample, see our [java/samples](https://github.com/sap-samples/cloud-cap-samples-java). [Learn more about Setting Up Local Development.](../../java/getting-started#local){.learn-more}

In addition, you need to prepare the following: #### 1. SAP BTP with SAP HANA Cloud Database up and Running {#btp-and-hana} - Access to [SAP BTP, for example a trial](https://developers.sap.com/tutorials/hcp-create-trial-account.html) - An [SAP HANA Cloud database running](https://help.sap.com/docs/hana-cloud/sap-hana-cloud-administration-guide/create-sap-hana-database-instance-using-sap-hana-cloud-central) in your subaccount - Entitlement for [`hdi-shared` service plan](https://help.sap.com/docs/hana-cloud/sap-hana-cloud-getting-started-guide/set-up-schema-or-hdi-container-cloud-foundry) for your subaccount - A [Cloud Foundry space](https://help.sap.com/docs/btp/sap-business-technology-platform/create-spaces?version=Cloud) ::: tip Starting the SAP HANA database takes several minutes Therefore, we recommend doing these steps early on. In trial accounts, you need to start the database **every day**. ::: #### 2. Latest Versions of `@sap/cds-dk` {#latest-cds} Ensure you have the latest version of `@sap/cds-dk` installed globally: ```sh npm -g outdated #> check whether @sap/cds-dk is listed npm i -g @sap/cds-dk #> if necessary ```

Likewise, ensure the latest version of `@sap/cds` is installed in your project: ```sh npm outdated #> check whether @sap/cds is listed npm i @sap/cds #> if necessary ```

#### 3. Cloud MTA Build Tool {#mbt} - Run `mbt` in a terminal to check whether you've installed it. - If not, install it according to the [MTA Build Tool's documentation](https://sap.github.io/cloud-mta-build-tool/download). - For macOS/Linux machines best is to install using `npm`: ```sh npm i -g mbt ``` - For Windows, [please also install `GNU Make`](https://sap.github.io/cloud-mta-build-tool/makefile/). #### 4. Cloud Foundry CLI w/ MTA Plugins {#cf-cli} - Run `cf -v` in a terminal to check whether you've installed version **8** or higher. - If not, install or update it according to the [Cloud Foundry CLI documentation](https://github.com/cloudfoundry/cli#downloads). - In addition, ensure to have the [MTA plugin for the Cloud Foundry CLI](https://github.com/cloudfoundry-incubator/multiapps-cli-plugin/tree/master/README.md) installed. ```sh cf add-plugin-repo CF-Community https://plugins.cloudfoundry.org cf install-plugin multiapps cf install-plugin html5-plugin ``` ## Prepare for Production {#prepare-for-production} If you followed CAP's grow-as-you-go approach so far, you've developed your application with an in-memory database and basic/mock authentication. To prepare for production you need to ensure respective production-grade choices are configured: ![You need to add SAP HANA Cloud, an App Router and XSUAA.](assets/deploy-overview.drawio.svg){} We'll use the `cds add ` CLI command for that, which ensures the required services are configured correctly and corresponding package dependencies are added to your _package.json_. ### 1. Using SAP HANA Database

While we used SQLite as a low-cost stand-in during development, we're going to use a managed SAP HANA database for production:

While we used SQLite or H2 as a low-cost stand-in during development, we're going to use a managed SAP HANA database for production:

```sh cds add hana --for production ``` [Learn more about using SAP HANA for production.](../databases-hana){.learn-more} ### 2. Using XSUAA-Based Authentication Configure your app for XSUAA-based authentication: ```sh cds add xsuaa --for production ``` ::: tip This will also generate an `xs-security.json` file The roles/scopes are derived from authorization-related annotations in your CDS models. Ensure to rerun `cds compile --to xsuaa`, as documented in the [_Authorization_ guide](/guides/security/authorization#xsuaa-configuration) whenever there are changes to these annotations. ::: ::: details For trial and extension landscapes, OAuth configuration is required Add the following snippet to your _xs-security.json_ and adapt it to the landscape you're deploying to: ```json "oauth2-configuration": { "redirect-uris": ["https://*.cfapps.us10-001.hana.ondemand.com/**"] } ``` ::: [Learn more about SAP Authorization and Trust Management/XSUAA.](https://discovery-center.cloud.sap/serviceCatalog/authorization-and-trust-management-service?region=all){.learn-more} ### 3. Using MTA-Based Deployment { #add-mta-yaml} We'll be using the [Cloud MTA Build Tool](https://sap.github.io/cloud-mta-build-tool/) to execute the deployment. The modules and services are configured in an `mta.yaml` deployment descriptor file, which we generate with: ```sh cds add mta ``` [Learn more about MTA-based deployment.](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/d04fc0e2ad894545aebfd7126384307c.html?locale=en-US){.learn-more} ### 4. Using App Router as Gateway { #add-app-router} The _App Router_ acts as a single point-of-entry gateway to route requests to. In particular, it ensures user login and authentication in combination with XSUAA. Two deployment options are available: - **Managed App Router**: for SAP Build Work Zone, the Managed App Router provided by SAP Fiori Launchpad is available. See the [end-to-end tutorial](https://developers.sap.com/tutorials/integrate-with-work-zone.html) for the necessary configuration in `mta.yaml` and on each _SAP Fiori application_. - **Custom App Router**: for scenarios without SAP Fiori Launchpad, the App Router needs to be deployed along with your application. Use the following command to enhance the application configuration: ```sh cds add approuter ``` [Learn more about the SAP BTP Application Router.](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/01c5f9ba7d6847aaaf069d153b981b51.html?locale=en-US){.learn-more} ### 5. User Interfaces { #add-ui } #### SAP Cloud Portal If you intend to deploy user interface applications, you also need to set up the [HTML5 Application Repository](https://discovery-center.cloud.sap/serviceCatalog/html5-application-repository-service) in combination with the [SAP Cloud Portal service](https://discovery-center.cloud.sap/serviceCatalog/cloud-portal-service): ```sh cds add portal ``` #### SAP Build Work Zone, Standard Edition For **single-tenant applications**, you can use [SAP Build Work Zone, Standard Edition](https://discovery-center.cloud.sap/serviceCatalog/sap-build-work-zone-standard-edition): ```sh cds add workzone ``` ### 6. Optional: Add Multitenancy { #add-multitenancy } To enable multitenancy for production, run the following command: ```sh cds add multitenancy --for production ``` > If necessary, modifies deployment descriptors such as _mta.yaml_ for Cloud Foundry. [Learn more about MTX services.](../multitenancy/#behind-the-scenes){.learn-more} ::: tip You're set! The previous steps are required _only once_ in a project's lifetime. With that done, we can repeatedly deploy the application. :::
### 7. Freeze Dependencies { #freeze-dependencies }

Deployed applications should freeze all their dependencies, including transient ones. Create a _package-lock.json_ file for that: ```sh npm update --package-lock-only ```

If you use multitenancy, also freeze dependencies for the MTX sidecar: ```sh npm update --package-lock-only --prefix mtx/sidecar ``` In addition, you need install and freeze dependencies for your UI applications: ```sh npm i --prefix app/browse npm i --prefix app/admin-books ``` [Learn more about dependency management for Node.js](../../node.js/best-practices#dependencies){.learn-more} ::: tip Regularly update your `package-lock.json` to consume latest versions and bug fixes Do so by running this command again, for example each time you deploy a new version of your application. ::: ## Build & Assemble { #build-mta } ### Build Deployables with `cds build` Run `cds build` to generate additional deployment artifacts and prepare everything for production in a local `./gen` folder as a staging area. While `cds build` is included in the next step `mbt build`, you can also run it selectively as a test, and to inspect what is generated: ```sh cds build --production ``` [Learn more about running and customizing `cds build`.](custom-builds){.learn-more} ### Assemble with `mbt build` ::: info Prepare monorepo setups The CAP samples repository on GitHub has a more advanced (monorepo) structure, so tell the `mbt` tool to find the `package-lock.json` on top-level: ```sh ln -sf ../package-lock.json ``` ::: Now, we use the `mbt` build tool to assemble everything into a single `mta.tar` archive: ```sh mbt build -t gen --mtar mta.tar ``` [Got errors? See the troubleshooting guide.](../../get-started/troubleshooting#mta){.learn-more} [Learn how to reduce the MTA archive size during development.](../../get-started/troubleshooting#reduce-mta-size){.learn-more} ## Deploy to Cloud {#deploy} Finally, we can deploy the generated archive to Cloud Foundry: ```sh cf deploy gen/mta.tar ``` [You need to be logged in to Cloud Foundry.](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/7a37d66c2e7d401db4980db0cd74aa6b.html){.learn-more} This process can take some minutes and finally creates a log output like this: ```log […] Application "bookshop" started and available at "[org]-[space]-bookshop.landscape-domain.com" […] ``` Copy and open this URL in your web browser. It's the URL of your App Router application. ::: tip For multitenant applications, you have to subscribe a tenant first In this case, the application is accessible via a tenant-specific URL after onboarding. ::: ### Inspect Apps in BTP Cockpit Visit the "Applications" section in your [SAP BTP cockpit](https://help.sap.com/docs/BTP/65de2977205c403bbc107264b8eccf4b/144e1733d0d64d58a7176e817fa6aeb3.html) to see the deployed apps: ![The screenshot shows the SAP BTP cockpit, when a user navigates to his dev space in the trial account and looks at all deployed applications.](./assets/apps-cockpit.png){.mute-dark} ::: tip Assign the _admin_ role We didn't do the _admin_ role assignment for the `AdminService`. You need to create a role collection and [assign the role and your user](https://developers.sap.com/tutorials/btp-app-role-assignment.html) to get access. ::: [Got errors? See the troubleshooting guide.](../../get-started/troubleshooting#cflogs-recent){.learn-more} ### Upgrade Tenants {.java} The CAP Java SDK offers `main` methods for Subscribe/Unsubscribe in the classes `com.sap.cds.framework.spring.utils.Subscribe/Unsubscribe` that can be called from the command line. This way, you can run the tenant subscribe/unsubscribe for the specified tenant. This would trigger also your custom handlers, which is useful for the local testing scenarios. In order to register all handlers of the application properly during the execution of a tenant operation `main` method, the component scan package must be configured. To set the component scan, the property cds.multitenancy.component-scan must be set to the package name of your application. The handler registration provides additional information that is used for the tenant subscribe, for example, messaging subscriptions that are created. ::: warning You can stop the CAP Java back end when you call this method, but the _MTX Sidecar_ application must be running! ::: This synchronization can also be automated, for example using [Cloud Foundry Tasks](https://docs.cloudfoundry.org/devguide/using-tasks.html) on SAP BTP and [Module Hooks](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/b9245ba90aa14681a416065df8e8c593.html) in your MTA. The `main` method optionally takes tenant ID (string) as the first input argument and tenant options (JSON string) as the second input argument. Alternatively, you can use the environment variables `MTCOMMAND_TENANTS` and `MTCOMMAND_OPTIONS` instead of arguments. The command-line arguments have higher priority, so you can use them to override the environment variables. The method returns the following exit codes. | Exit Code | Result | | --------- | ------------------------------------------------------------------------------------------------ | | 0 | Tenant subscribed/unsubscribed successfully. | | 3 | Failed to subscribe/unsubscribe the tenant. Rerun the procedure to make sure the tenant is subscribed/unsubscribed. | To run this method locally, use the following command where `` is the one of your applications: ::: code-group ```sh [>= Spring Boot 3.2.0] java -cp -Dloader.main=com.sap.cds.framework.spring.utils.Subscribe/Unsubscribe org.springframework.boot.loader.launch.PropertiesLauncher [] ``` ```sh [< Spring Boot 3.2.0] java -cp -Dloader.main=com.sap.cds.framework.spring.utils.Subscribe/Unsubscribe org.springframework.boot.loader.PropertiesLauncher [] ``` ::: In the SAP BTP, Cloud Foundry environment, it can be tricky to construct such a command. The reason is that the JAR file is extracted by the Java buildpack and the place of the Java executable isn't easy to determine. Also the place differs for different Java versions. Therefore, we recommend adapting the start command that is generated by the buildpack and run the adapted command: ::: code-group ```sh [>= Spring Boot 3.2.0] sed -i 's/org.springframework.boot.loader.launch.JarLauncher/org.springframework.boot.loader.launch.PropertiesLauncher/g' /home/vcap/staging_info.yml && sed -i 's/-Dsun.net.inetaddr.negative.ttl=0/-Dsun.net.inetaddr.negative.ttl=0 -Dloader.main=com.sap.cds.framework.spring.utils.Subscribe/Unsubscribe/g' /home/vcap/staging_info.yml && jq -r .start_command /home/vcap/staging_info.yml | sed 's/^/ MTCOMMAND_TENANTS=my-tenant [MTCOMMAND_TENANTS=]/' | bash ``` ```sh [< Spring Boot 3.2.0] sed -i 's/org.springframework.boot.loader.JarLauncher/org.springframework.boot.loader.PropertiesLauncher/g' /home/vcap/staging_info.yml && sed -i 's/-Dsun.net.inetaddr.negative.ttl=0/-Dsun.net.inetaddr.negative.ttl=0 -Dloader.main=com.sap.cds.framework.spring.utils.Subscribe/Unsubscribe/g' /home/vcap/staging_info.yml && jq -r .start_command /home/vcap/staging_info.yml | sed 's/^/ MTCOMMAND_TENANTS=my-tenant [MTCOMMAND_TENANTS=]/' | bash ``` ```sh [Java 8] sed -i 's/org.springframework.boot.loader.JarLauncher/-Dloader.main=com.sap.cds.framework.spring.utils.Subscribe/Unsubscribe org.springframework.boot.loader.PropertiesLauncher/g' /home/vcap/staging_info.yml && jq -r .start_command /home/vcap/staging_info.yml | sed 's/^/ MTCOMMAND_TENANTS=my-tenant [MTCOMMAND_TENANTS=]/' | bash ``` ::: --- {} # Appendices ## Deploy using `cf push` As an alternative to MTA-based deployment, you can choose Cloud Foundry-native deployment using [`cf push`](https://docs.cloudfoundry.org/devguide/push.html), or `cf create-service-push` respectively. ### Prerequisites Install the [_Create-Service-Push_ plugin](https://github.com/dawu415/CF-CLI-Create-Service-Push-Plugin): ```sh cf install-plugin Create-Service-Push ``` This plugin acts the same way as `cf push`, but extends it such that services are _created_ first. With the plain `cf push` command, this is not possible. ### Add a `manifest.yml` {#add-manifest} ```sh cds add cf-manifest ``` This creates two files, a _manifest.yml_ and _services-manifest.yml_ in the project root folder. - _manifest.yml_ holds the applications. In the default layout, one application is the actual server holding the service implementations, and the other one is a 'DB deployer' application, whose sole purpose is to start the SAP HANA deployment. - _services-manifest.yml_ defines which Cloud Foundry services shall be created. The services are derived from the service bindings in _package.json_ using the [`cds.requires` configuration](../../node.js/cds-env#services). ::: tip Version-control manifest files Unlike the files in the _gen_ folders, these manifest files are genuine sources and should be added to the version control system. This way, you can adjust them to your needs as you evolve your application. ::: ### Build the Project This prepares everything for deployment, and -- by default -- writes the build output, that is the deployment artifacts, to folder _./gen_ in your project root. ```sh cds build --production ``` [Learn how `cds build` can be configured.](custom-builds#build-config){.learn-more} The `--production` parameter ensures that the cloud deployment-related artifacts are created by `cds build`. See section [SAP HANA database deployment](../databases-hana) for more details. ### Push the Application { #push-the-application} This command creates service instances, pushes the applications and binds the services to the application with a single call: ```sh cf create-service-push ``` During deployment, the plugin reads the _services-manifest.yml_ file and creates the services listed there. It then reads _manifest.yml_, pushes the applications defined there, and binds these applications to service instances created before. If the service instances already exist, only the `cf push` operation will be executed. You can also apply some shortcuts: - Use `cf push` directly to deploy either all applications, or `cf push ` to deploy a single application. - Use `cf create-service-push --no-push` to only create or update service-related data without pushing the applications. In the deployment log, find the application URL in the `routes` line at the end: ```log{3} name: bookshop-srv requested state: started routes: bookshop-srv.cfapps.sap.hana.ondemand.com ``` Open this URL in the browser and try out the provided links, for example, `…/browse/Books`. Application data is fetched from SAP HANA. ::: tip Ensure successful SAP HANA deployment Check the deployment logs of the database deployer application using ```sh cf logs -db-deployer --recent ``` to ensure that SAP HANA deployment was successful. The application itself is by default in state `started` after HDI deployment has finished, even if the HDI deployer returned an error. To save resources, you can explicitly stop the deployer application afterwards. ::: ::: tip No Fiori preview in the cloud The [SAP Fiori Preview](../../advanced/fiori#sap-fiori-preview), that you are used to see from local development, is only available for the development profile and not available in the cloud. For productive applications, you should add a proper SAP Fiori application. ::: ::: warning Multitenant applications are not supported yet as multitenancy-related settings are not added to the generated descriptors. The data has to be entered manually. ::: [Got errors? See the troubleshooting guide.](../../get-started/troubleshooting#aborted-deployment-with-the-create-service-push-plugin){.learn-more} # Deploy to Kyma Runtime You can run your CAP application in the [Kyma Runtime](https://discovery-center.cloud.sap/serviceCatalog/kyma-runtime?region=all). This runtime of the SAP Business Technology Platform is the SAP managed offering for the [Kyma project](https://kyma-project.io/). This guide helps you to run your CAP applications on SAP BTP Kyma Runtime. ## Overview As well as Kubernetes, Kyma is a platform to run containerized workloads. The service's files are provided as a container image, commonly referred to as a Docker image. In addition, the containers to be run on Kubernetes, their configuration and everything else that is needed to run them, are described by Kubernetes resources. In consequence, two kinds of artifacts are needed to run applications on Kubernetes: 1. Container images 2. Kubernetes resources The following diagram shows the steps to run on the SAP BTP Kyma Runtime: ![A CAP Helm chart is added to your project. Then you built your project as container images and push those images to a container registry of your choice. As last step the Helm chart is deployed to your Kyma resources, where service instances of SAP BTP services are created and pods pull the previously created container images from the container registry.](assets/deploy-kyma.drawio.svg) 1. [**Add** a Helm chart](#cds-add-helm) 2. [**Build** container images](#build-images) 3. [**Deploy** your application by applying Kubernetes resources](#deploy-helm-chart) ## Prerequisites {#prerequisites} + You prepared your project as described in the [Deploy to Cloud Foundry](to-cf) guide. + Use a Kyma enabled [Trial Account](https://account.hanatrial.ondemand.com/) or [learn how to get access to a Kyma cluster](#get-access-to-a-cluster). + You need a [Container Image Registry](#get-access-to-a-container-registry) + Get the required SAP BTP service entitlements + Download and install the following command line tools: + [`kubectl` command line client](https://kubernetes.io/docs/tasks/tools/) for Kubernetes + [Docker Desktop or Docker for Linux](https://docs.docker.com/get-docker/) + [`pack` command line tool](https://buildpacks.io/docs/tools/pack/) + [`helm` command line tool](https://helm.sh/docs/intro/install/) + [`ctz` command line tool](https://www.npmjs.com/package/ctz) ::: warning Make yourself familiar with Kyma and Kubernetes. CAP doesn't provide consulting on it. ::: ## Prepare for Production The detailed procedure is described in the [Deploy to Cloud Foundry guide](to-cf#prepare-for-production). Run this command to fast-forward: ```sh cds add hana,xsuaa --for production ``` ## Add Helm Chart {#cds-add-helm} CAP provides a configurable [Helm chart](https://helm.sh/) for Node.js and Java applications. ```sh cds add helm ``` This command adds the Helm chart to the _chart_ folder of your project with 3 files: `values.yaml`, `Chart.yaml` and `values.schema.json`. During cds build, the _gen_/_chart_ folder is generated. This folder will have all the necessary files required to deploy the helm chart. Files from the _chart_ folder in root of the project are copied to the folder generated in _gen_ folder. The files in the _gen/chart_ folder support the deployment of your CAP service, database and UI content, and the creation of instances for BTP services. [Learn more about CAP Helm chart.](#about-cap-helm){.learn-more} ## Build Images {#build-images} We'll be using the [Containerize Build Tool](https://www.npmjs.com/package/ctz/) to build the images. The modules are configured in a `containerize.yaml` descriptor file, which we generate with: ```sh cds add containerize ``` #### Configure Image Repository Specify the repository where you want to push the images: ```yaml ... repository: ``` ::: warning You should be logged in to the above repository to be able to push images to it. You can use `docker login -u ` to login. ::: Now, we use the `ctz` build tool to build all the images: ```sh ctz containerize.yaml ``` > This will start containerizing your modules based on the configuration in the specified file. After it is done, it will ask whether you want to push the images or not. Type `y` and press enter to push your images. You can also use the above command with `--push` flag to skip this. If you want more logs, you can use the `--log` flag with the above command. [Learn more about Containerize Build Tool](https://www.npmjs.com/package/ctz/){.learn-more} ### UI Deployment For UI access, you can use the standalone and the managed App Router as explained in [this blog](https://blogs.sap.com/2021/12/09/using-sap-application-router-with-kyma-runtime/). The `cds add helm` command [supports deployment](#html5-applications) to the [HTML5 application repository](https://help.sap.com/products/BTP/65de2977205c403bbc107264b8eccf4b/f8520f572a6445a7bfaff4a1bbcbe60a.html?locale=en-US&version=Cloud) which can be used with both options. For that, create a container image with your UI files configured with the [HTML5 application deployer](https://help.sap.com/docs/BTP/65de2977205c403bbc107264b8eccf4b/9b178ab3388c4647b0c52f2c85641844.html). The `cds add helm` command also supports deployment of standalone approuter. To configure backend destinations, have a look at the [approuter configuration section.](#configure-approuter-specifications) ## Deploy Helm Chart {#deploy-helm-chart} Once your Helm chart is created, your container images are uploaded to a registry and your cluster is prepared, you're almost set for deploying your Kyma application. ### Create Service Instances for SAP HANA Cloud {#hana-cloud-instance} 1. Enable SAP HANA for your project as explained in the [CAP guide for SAP HANA](../databases-hana). 2. Create an SAP HANA database. 3. To create HDI containers from Kyma, you need to [create a mapping between your namespace and SAP HANA Cloud instance](https://blogs.sap.com/2022/12/15/consuming-sap-hana-cloud-from-the-kyma-environment/). ::: warning Set trusted source IP addresses Make sure that your SAP HANA Cloud instance can be accessed from your Kyma cluster by [setting the trusted source IP addresses](https://help.sap.com/docs/HANA_CLOUD/9ae9104a46f74a6583ce5182e7fb20cb/0610e4440c7643b48d869a6376ccaecd.html). ::: ### Deploy using CAP Helm Chart Before deployment, you need to set the container image and cluster specific settings. #### Configure Access to Your Container Images Add your container image settings to your _chart/values.yaml_: ```yaml ... global: domain: imagePullSecret: name: image: registry: tag: latest ``` You can use the pre-configured domain name for your Kyma cluster: ```yaml kubectl get gateway -n kyma-system kyma-gateway \ -o jsonpath='{.spec.servers[0].hosts[0]}' ``` To use images on private container registries you need to [create an image pull secret](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). For image registry, use the same value you mentioned in `containerize.yaml` #### Configure Approuter Specifications By default `srv-api` and `mtx-api` (only in Multi Tenant Application) are configured. If you're using any other destination or your `xs-app.json` file has a different destination, update the destinations under the `backendDestinations` key in _values.yaml_ file: ```yaml backendDestinations: backend: service: srv ``` > `backend` is the name of the destination. `service` points to the deployment name whose url will be used for this destination. #### Deploy CAP Helm Chart 1. Execute `cds build --production` to generate the helm chart in _gen_ folder. 2. Deploy using `helm` command: ```sh helm upgrade --install bookshop ./gen/chart \ --namespace bookshop-namespace --create-namespace ``` This installs the Helm chart from the _gen/chart_ folder with the release name `bookshop` in the namespace `bookshop-namespace`. ::: tip With the `helm upgrade --install` command you can install a new chart as well as upgrade an existing chart. ::: This process can take a few minutes to complete and create the log output: ```log […] The release bookshop is installed in namespace [namespace]. Your services are available at: [workload] - https://bookshop-[workload]-[namespace].[configured-domain] […] ``` Copy and open this URL in your web browser. It's the URL of your application. ::: info If a standalone approuter is present, the srv and sidecar aren't exposed and only the approuter URL will be logged. But if an approuter isn't present then srv and sidecar are also exposed and their URL will also be logged. ::: [Learn more about using a private registry with your Kyma cluster.](#setup-your-cluster-for-a-private-container-registry){.learn-more} [Learn more about the CAP Helm chart settings](#configure-helm-chart){ .learn-more} [Learn more about using `helm upgrade`](https://helm.sh/docs/helm/helm_upgrade){ .learn-more} ::: tip Try out the [CAP SFLIGHT](https://github.com/SAP-samples/cap-sflight) and [CAP for Java](https://github.com/SAP-samples/cloud-cap-samples-java) examples on Kyma. ::: ## Customize Helm Chart {#customize-helm-chart} ### About CAP Helm Chart { #about-cap-helm} The following files are added to a _chart_ folder by executing `cds add helm`: | File/Pattern | Description | | --------------------- | ---------------------------------------------------------- | | _values.yaml_ | [Configuration](#configure-helm-chart) of the chart; The initial configuration is determined from your CAP project. | | _Chart.yaml_ | Chart metadata that is initially determined from the _package.json_ file | | _values.schema.json_ | JSON Schema for _values.yaml_ file | The following files are added to a _gen/chart_ folder along with all the files in the _chart_ folder in the root of the project by executing `cds build` after adding `helm`: | File/Pattern | Description | | --------------------- | ---------------------------------------------------------- | | _templates/*.tpl_ | Template libraries used in the template resources | | _templates/NOTES.txt_ | Message printed after installing or upgrading the Helm charts | | _templates/*.yaml_ | Template files for the Kubernetes resources | [Learn how to create a Helm chart from scratch from the Helm documentation.](https://helm.sh/docs){.learn-more} ### Configure {#configure-helm-chart} [CAP's Helm chart](#cds-add-helm) can be configured by the settings as explained below. Mandatory settings are marked with . You can change the configuration by editing the _chart/values.yaml_ file. When you call `cds add helm` again, your changes will be persisted and only missing default values are added. The `helm` CLI also offers you other options to overwrite settings from _chart/values.yaml_ file: + Overwrite properties using the `--set` parameter. + Overwrite properties from a YAML file using the `-f` parameter. ::: tip It is recommended to do the main configuration in the _chart/values.yaml_ file and have additional YAML files for specific deployment types (dev, test, productive) and targets. ::: #### Global Properties | Property | Description | Mandatory | | --------------- | ------------------------------------------------------------- | :---------: | | imagePullSecret → name | Name of secret to access the container registry | ( ) ¹ | | domain | Kubernetes cluster ingress domain (used for application URLs) | | | image → registry | Name of the container registry from where images are pulled | | ¹: Mandatory only for private docker registries #### Deployment Properties The following properties are available for the `srv` key: | Property | Description | Mandatory | |------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------:| | **bindings** | [Service Bindings](#configuration-options-for-service-bindings) | | | **resources** | [Kubernetes Container resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) | | | **env** | Map of additional env variables | | | **health** | [Kubernetes Liveness, Readyness and Startup Probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) | | | → liveness → path | Endpoint for liveness and startup probe | | | → readiness → path | Endpoint for readiness probe | | | → startupTimeout | Wait time in seconds until the health checks are started | | | **image** | [Container image](#configuration-options-for-container-images) | | You can explore more configuration options in the subchart's directory _gen/chart/charts/web-application_. #### SAP BTP Services The helm chart supports to create service instances for commonly used services. Services are pre-populated in the _chart/values.yaml_ file based on the used services in the `requires` section of the CAP configuration (for example, _package.json_) file. You can use the following services in your configuration: | Property | Description | Mandatory | |----------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|:---------:| | **xsuaa** | Enables the creation of a XSUAA service instance. See details for [Node.js](../../node.js/authentication) and [Java](../../java/security) projects. | | | parameters → xsappname | Name of XSUAA application. Overwrites the value from the _xs-security.json_ file. (unique per subaccount) | | | parameters → HTML5Runtime_enabled | Set to true for use with Launchpad Service | | | **connectivity** | Enables [on-premise connectivity](#connectivity-service) | | | **event-mesh** | Enables SAP Event Mesh; [messaging guide](../messaging/), [how to enable the SAP Event Mesh](../messaging/event-mesh) | | | **html5-apps-repo-host** | HTML5 Application Repository | | | **hana** | HDI Shared Container | | | **service-manager** | Service Manager Container | | | **saas-registry** | SaaS Registry Service | | [Learn how to configure services in your Helm chart](#configuration-options-for-services){.learn-more} #### SAP HANA The deployment job of your database content to a HDI container can be configured using the `hana-deployer` section with the following properties: | Property | Description | Mandatory | |---------------|------------------------------------------------------------------------------------------------------------------|:---------:| | **bindings** | [Service binding](#configuration-options-for-service-bindings) to the HDI container's secret | | | **image** | [Container image](#configuration-options-for-container-images) of the HDI deployer | | | **resources** | [Kubernetes Container resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) | | | **env** | Map of additional environment variables | | #### HTML5 Applications The deployment job of HTML5 applications can be configured using the `html5-apps-deployer` section with the following properties: [Container image]: #configuration-options-for-container-images [HTML5 application deployer]: https://help.sap.com/docs/BTP/65de2977205c403bbc107264b8eccf4b/9b178ab3388c4647b0c52f2c85641844.html [Kubernetes Container resources]: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ | Property | Description | Mandatory | |--------------------------|---------------------------------------------------------------------------------------------------------------------------------------|:---------:| | **image** | [Container image] of the [HTML5 application deployer] | | | **bindings** | [Service bindings](#configuration-options-for-service-bindings) to XSUAA, destinations and HTML5 Application Repository Host services | | | **resources** | [Kubernetes Container resources] | | | **env** | Map of additional environment variables | | | → SAP_CLOUD_SERVICE | Name for your business service (unique per subaccount) | | ::: tip Run `cds add html5-repo` to automate the setup for HTML5 application deployment. ::: #### Backend Destinations Backend destinations maybe required for HTML5 applications or for App Router deployment. They can be configured using the `backendDestinations` section with the following properties: | Property | Description | |------------------|-----------------------------------------------------| | (key) | Name of backend destination | | service: (value) | Value is the target Kubernetes service (like `srv`) | If you want to add an external destination, you can do so by providing the `external` property like this: ```yaml ... backendDestinations: srv-api: service: srv ui5: # [!code ++] external: true # [!code ++] name: ui5 # [!code ++] Type: HTTP # [!code ++] proxyType: Internet # [!code ++] url: https://ui5.sap.com # [!code ++] Authentication: NoAuthentication # [!code ++] ``` > Our helm chart will remove the `external` key and add the rest of the keys as-is to the environment variable. #### Connectivity Service Use `cds add connectivity`, to add a volume to your `srv` deployment. ::: warning Create an instance of the SAP BTP Connectivity service with plan `connectivity_proxy` and a service binding, before deploying the first application that requires it. Using this plan, a proxy to the connectivity service gets installed into your Kyma cluster. This may take a few minutes. The connectivity proxy uses the first created instance in a cluster for authentication. This instance must not be deleted as long as connectivity is used. ::: The volume you've added to your `srv` deployment is needed, to add additional connection information, compared to what's available from the service binding. ```yaml srv: ... additionalVolumes: - name: connectivity-secret volumeMount: mountPath: /bindings/connectivity readOnly: true projected: sources: - secret: name: optional: false - secret: name: optional: false items: - key: token_service_url path: url - configMap: name: "RELEASE-NAME-connectivity-proxy-info" optional: false ``` In the volumes added, replace the value of `` with the binding that you created earlier. If the binding is created in a different namespace then you need to create a secret with details from the binding and use that secret. ::: tip You don't have to edit `RELEASE-NAME` in the `configMap` property. It is passed as a template string and will be replaced with your actual release name by Helm. ::: #### Arbitrary Service These are the steps to create and bind to an arbitrary service, using the binding of the feature toggle service to the CAP application as an example: 1. In the _chart/Chart.yaml_ file, add an entry to the `dependencies` array. ```yaml dependencies: ... - name: service-instance alias: feature-flags version: 0.1.0 ``` 2. Add the service configuration and the binding in the _chart/values.yaml_ file: ```yaml feature-flags: serviceOfferingName: feature-flags servicePlanName: lite ... srv: bindings: feature-flags: serviceInstanceName: feature-flags ``` > The `alias` property in the `dependencies` array must match the property added in the root of _chart/values.yaml_ and the value of `serviceInstanceName` in the binding. ::: warning There should be at least one service instance created by `cds add helm` if you want to bind an arbitrary service. ::: #### Configuration Options for Services _Services have the following configuration options:_ | Property | Type | Description | Mandatory | ------------------- | ----------- | ---------------------------------------- |:-----: | | **fullNameOverride** | string | Use instead of the generated name | | **serviceOfferingName** | string | Technical service offering name from service catalog | | **servicePlanName** | string | Technical service plan name from service catalog |

CAP config	VCAP_SERVICES
```json { "cds": { "requires": { "db": { ... } } } } ```	```json { "VCAP_SERVICES": { "hana": [{ "name": "db", ... }] } } ```
```json { "cds": { "requires": { "db": { "kind": "hana" } } } } ```	```json { "VCAP_SERVICES": { "hana": [{ "label": "hana", ... }] } } ```
```json { "cds": { "requires": { "db": { "vcap": { "name": "myDb" } } } } } ```	```json { "VCAP_SERVICES": { "hana": [{ "name": "myDb", ... }] } } ```

Predicate	Description	Example
EQ	Test if this value equals a given value. NULL values might be treated as unknown resulting in a three-valued logic as in SQL.	`Select.from("bookshop.Books") .where(b -> b.get("stock") .eq(15));`
NE	Test if this value is NOT equal to a given value. NULL values might be treated as unknown resulting in a three-valued logic as in SQL.	`Select.from("bookshop.Books") .where(b -> b.get("stock") .ne(25));`
IS	Test if this value equals a given value. NULL values are treated as any other value.	`Select.from("bookshop.Books") .where(b -> b.get("stock") .is(15));`
IS NOT	Test if this value is NOT equal to a given value. NULL values are treated as any other value.	`Select.from("bookshop.Books") .where(b -> b.get("stock") .isNot(25));`
GT	Test if this value is greater than a given value.	`Select.from("bookshop.Books") .where(b -> b.get("stock") .gt(5));`
LT	Test if this value is less than a given value.	`Select.from("bookshop.Books") .where(b -> b.get("stock") .lt(5));`
LE	Test if this value is less than or equal to a given value.	`Select.from("bookshop.Books") .where(b -> b.get("stock") .le(5));`
BETWEEN	Test if this value is between¹ a range of values.	`Select.from("bookshop.Books") .where(b -> b.get("stock") .between(5, 10));`

Operator	Description	Example
CONTAINS	Test if this string value contains a given substring.	`Select.from(EMPLOYEE) .where(e -> e.name() .contains("oni"));`
STARTS WITH	Test if this string value starts with a given prefix.	`Select.from("bookshop.Books") .where(b -> b.get("title") .startsWith("The"));`
ENDS WITH	Test if this string value ends with a given suffix.	`Select.from("bookshop.Books") .where(b -> b.get("title") .endsWith("Raven"));`

Operator	Description	Example
AND	Returns a predicate that represents a logical AND of this predicate and another.	`Select.from("bookshop.Authors") .where(a -> a.get("name").eq("Peter) .and(a.get("Id").eq(1)));`
OR	Returns a predicate that represents a logical OR of this predicate and another.	`Select.from("bookshop.Authors") .where(a -> a.get("name").eq("Peter) .or(a.get("Id").eq(1)));`
NOT	Returns a predicate that represents the logical negation of this predicate.	`Select.from("bookshop.Authors") .where(a -> not(a.get("Id").eq(3)));`