Transactional Outbox
Usually the emit of messages should be delayed until the main transaction succeeded. Otherwise recipients will also receive messages in case of a rollback. To solve this problem, an outbox is used internally to defer the emit of messages until the success of the current transaction.
Persistent Outbox (Default)
Using the persistent outbox, the to-be-emitted message is stored in a database table first. The same database transaction is used as for other operations, therefore transactional consistency is guaranteed.
The persistent outbox is globally enabled for all deferrable services (for example for cds.MessagingService and cds.AuditLogService
). You can set the global outbox configuration, the defaults are:
{
"requires": {
"outbox": {
"kind": "persistent-outbox",
"maxAttempts": 20,
"chunkSize": 100,
"storeLastError": true,
"parallel": true
}
}
}
{
"requires": {
"outbox": {
"kind": "persistent-outbox",
"maxAttempts": 20,
"chunkSize": 100,
"storeLastError": true,
"parallel": true
}
}
}
The optional parameters are:
maxAttempts
(default20
): The number of unsuccessful emits until the message is ignored. It will still remain in the database table.chunkSize
(default100
): The number of messages which are read from the database table in one go.storeLastError
(defaulttrue
): Specifies if error information of the last failed emit should be stored in the outbox table.parallel
(defaulttrue
): Specifies if messages are sent in parallel (faster but the order isn't guaranteed).
Once the transaction succeeds, the messages are read from the database table and emitted. If an emit was successful, the respective message is deleted from the database table. If not, there will be retries after (exponentially growing) waiting times. After a maximum number of attempts, the message is ignored for processing and remains in the database table which therefore also acts as a dead letter queue. There is only one active message processor per service, tenant and app instance, hence there won't be duplicate emits except in the unlikely case of an app crash right after the emit and before the deletion of the message entry.
TIP
Some errors during the emit are identified as unrecoverable, for example in SAP Event Mesh if the used topic is forbidden. The respective message is then updated and the attempts
field is set to maxAttempts
to prevent further processing. Programming errors crash the server instance and must be fixed.
Your database model is automatically extended by the entity cds.outbox.Messages
, as follows:
using cuid from '@sap/cds/common';
namespace cds.outbox;
entity Messages : cuid {
timestamp: Timestamp;
target: String;
msg: LargeString;
attempts: Integer default 0;
partition: Integer default 0;
lastError: LargeString;
lastAttemptTimestamp: Timestamp @cds.on.update : $now;
}
using cuid from '@sap/cds/common';
namespace cds.outbox;
entity Messages : cuid {
timestamp: Timestamp;
target: String;
msg: LargeString;
attempts: Integer default 0;
partition: Integer default 0;
lastError: LargeString;
lastAttemptTimestamp: Timestamp @cds.on.update : $now;
}
TIP
In your CDS model, you can refer to the entity cds.outbox.Messages
using the path @sap/cds/srv/outbox
, for example to expose it in a service.
WARNING
- If the app crashes, another emit for the respective tenant and service is necessary to restart the message processing.
- The user id is stored to recreate the correct context.
To overwrite the outbox configuration for a particular service, you can specify the outbox
option.
Example:
{
"requires": {
"messaging": {
"kind": "enterprise-messaging",
"outbox": {
"maxAttempts": 10,
"chunkSize": 10
}
}
}
}
{
"requires": {
"messaging": {
"kind": "enterprise-messaging",
"outbox": {
"maxAttempts": 10,
"chunkSize": 10
}
}
}
}
In-Memory Outbox
Messages are emitted when the current transaction is successful. Until then, messages are only kept in memory. This is similar to the following code if done manually:
cds.context.on('succeeded', () => this.emit(msg))
cds.context.on('succeeded', () => this.emit(msg))
WARNING
The message is lost if its emit fails, there is no retry mechanism. The app will crash if the error is identified as unrecoverable, for example in SAP Event Mesh if the used topic is forbidden.
Immediate Emit
To disable deferred emitting for a particular service, you can set the outbox
option of your service to false
:
{
"requires": {
"messaging": {
"kind": "enterprise-messaging",
"outbox": false
}
}
}
{
"requires": {
"messaging": {
"kind": "enterprise-messaging",
"outbox": false
}
}
}
Troubleshooting
Delete Entries in the Outbox Table
To manually delete entries in the table cds.outbox.Messages
, you can either expose it in a service or programmatically modify it using the cds.outbox.Messages
entity:
const db = await cds.connect.to('db')
const { Messages } = db.entities('cds.outbox')
await DELETE.from(Messages)
const db = await cds.connect.to('db')
const { Messages } = db.entities('cds.outbox')
await DELETE.from(Messages)
Outbox Table Not Found
If the outbox table is not found on the database, this can be caused by insufficient configuration data in package.json.
In case you have overwritten requires.db.model
there, make sure to add the outbox model path @sap/cds/srv/outbox
:
"requires": {
"db": { ...
"model": [..., "@sap/cds/srv/outbox"]
}
}
"requires": {
"db": { ...
"model": [..., "@sap/cds/srv/outbox"]
}
}
The following is only relevant if you're using @sap/cds version < 6.7.0 and you've configured options.model
in custom build tasks. Add the model path accordingly:
"build": {
"tasks": [{ ...
"options": { "model": [..., "@sap/cds/srv/outbox"] }
}]
}
"build": {
"tasks": [{ ...
"options": { "model": [..., "@sap/cds/srv/outbox"] }
}]
}
Note that model configuration isn't required for CAP projects using the standard project layout that contain the folders db
, srv
, and app
. In this case, you can delete the entire model
configuration.