Migration from Old MTX ​

Towards new multitenancy capabilities

This guide is available for Node.js and Java. Press v to switch, or use the toggle.

Functional Differences ​

Before you start to migrate to @sap/cds-mtxs, read about the differences compared to the old MTX.

Persistence Changes ​

With @sap/cds-mtxs, the persistence has been simplified. There's no second container needed (META-tenant) any longer. Instead, tenant-specific metadata, such as extensions, are stored in the same container as the application data.

In addition, @sap/cds-mtxs also uses a dedicated tenant t0 to store some runtime data, such as job logs.

Extensibility ​

Changes of Extension Persistence ​

In contrast to @sap/cds-mtx, with @sap/cds-mtxs, the extensions are no longer stored as sources, but only as compiled csn files. Instead of running a build on the server with each extension activation, the build is now run locally before the extension is deployed. The extensions are then stored as csn files with a tag as key. When using cds push, the tag is derived from the name of the extension project in package.json.

Example package.json of extension project:

{
  "name": "@capire/orders-ext",
  "extends": "@capire/orders",
  ...
}

{
  "name": "@capire/orders-ext",
  "extends": "@capire/orders",
  ...
}

When the extension is pushed, it is stored with the tag @capire/orders-ext.

Also check the Push API.

Handling of extension sources ​

As mentioned previously, cds push only uploads compiled extensions as csn documents. That means, it's no longer possible to download the sources from the server.
Source control is expected to be done by the SaaS application provider using his own source repository. See also Extension project templates

Security ​

Some of the roles have changed with @sap/cds-mtxs.

Permanent and Temporary Limitations ​

Temporary Limitations ​

• Diagnose API isn't available.
• Upload of extension only works synchronously.

Permanent Limitations ​

• Scopes aren't configurable.

• It isn't possible to have tenant-specific model versions.

• Use of SAP HANA hdbmigrationtable is only possible for entities that aren't to be extended.

• Upload of arbitrary custom files together with extensions is no longer available.

Migration Steps ​

To switch to @sap/cds-mtxs, you need to change your project configuration, your custom handlers, and you might need to update the database content.

Adapt Project Configuration ​

"requires": {
  "multitenancy": true,
  "extensibility": true
}

"requires": {
  "multitenancy": true,
  "extensibility": true
}

cds add multitenancy

cds add multitenancy

# build-parameters:
#   before-all:
#     - builder: custom
#       commands:
#         - npm install --production
#         - npx -p @sap/cds-dk cds build --production

# build-parameters:
#   before-all:
#     - builder: custom
#       commands:
#         - npm install --production
#         - npx -p @sap/cds-dk cds build --production

requires:
  ...
  - name: sidecar
    properties:
      CDS_MULTITENANCY_MTXS_ENABLED: true
      CDS_MULTITENANCY_SIDECAR_URL: ~{url}

requires:
  ...
  - name: sidecar
    properties:
      CDS_MULTITENANCY_MTXS_ENABLED: true
      CDS_MULTITENANCY_SIDECAR_URL: ~{url}

modules:
  - name: bookshop-mtx
    type: nodejs
    path: mtx/sidecar # adapted path
    build-parameters: # added build section
      builder: custom
      build-result: gen
      commands:
        - npm run build
      requires:
        - name: bookshop-srv
    parameters:
      memory: 256M
      disk-quota: 1G
    requires:
      - name: bookshop-auth
      - name: bookshop-db
    provides:
      - name: mtx-api
        properties:
          mtx-url: ${default-url}

modules:
  - name: bookshop-mtx
    type: nodejs
    path: mtx/sidecar # adapted path
    build-parameters: # added build section
      builder: custom
      build-result: gen
      commands:
        - npm run build
      requires:
        - name: bookshop-srv
    parameters:
      memory: 256M
      disk-quota: 1G
    requires:
      - name: bookshop-auth
      - name: bookshop-db
    provides:
      - name: mtx-api
        properties:
          mtx-url: ${default-url}

"workspaces": [
  "mtx/sidecar"
]

"workspaces": [
  "mtx/sidecar"
]

npm i --package-lock-only --prefix mtx/sidecar

npm i --package-lock-only --prefix mtx/sidecar

{
  "build": {
    "target": "."
  },
  "profiles": ["with-mtx-sidecar", "java"],
  "requires": {
    "multitenancy": true
  }
}

{
  "build": {
    "target": "."
  },
  "profiles": ["with-mtx-sidecar", "java"],
  "requires": {
    "multitenancy": true
  }
}

"requires": {
    "multitenancy": true,
    "extensibility": true
  }

"requires": {
    "multitenancy": true,
    "extensibility": true
  }

Security Adaptations ​

The scopes needed by extension developers have changed. Scopes ExtendCDS and ExtendCDSdelete have changed to cds.ExtensionDeveloper. Make sure to adapt all occurrences in your security configuration (xs-security.json).

Communicate to customer admins and extension developers to add the new scope to their role collection. Also adjust the documentation for the SaaS application accordingly if available.

cds.on('served', async () => {
  const { 'cds.xt.SaasProvisioningService': provisioning } = cds.services
  const { 'cds.xt.DeploymentService': deployment } = cds.services

  await provisioning.prepend(() => {
    provisioning.on('UPDATE', 'tenant', async (req, next) => { ... })
    provisioning.on('dependencies', async (req, next) => { ... })
    ...
  })
  await deployment.prepend(() => {
    // previously this was `upgradeTenant`
    deployment.on('upgrade', async (req) => {
      // HDI container credentials are not yet available here
    })
    // previously this was `deployToDb`
    deployment.on('deploy', async (req) => {
      const { tenant, options: { container } } = req.data
      ...
    })
    ...
  })
})

cds.on('served', async () => {
  const { 'cds.xt.SaasProvisioningService': provisioning } = cds.services
  const { 'cds.xt.DeploymentService': deployment } = cds.services

  await provisioning.prepend(() => {
    provisioning.on('UPDATE', 'tenant', async (req, next) => { ... })
    provisioning.on('dependencies', async (req, next) => { ... })
    ...
  })
  await deployment.prepend(() => {
    // previously this was `upgradeTenant`
    deployment.on('upgrade', async (req) => {
      // HDI container credentials are not yet available here
    })
    // previously this was `deployToDb`
    deployment.on('deploy', async (req) => {
      const { tenant, options: { container } } = req.data
      ...
    })
    ...
  })
})

parameters:
  service: saas-registry
  config:
    appUrls:
      getDependencies: ~{mtx-api/mtx-url}/-/cds/saas-provisioning/dependencies
      onSubscription: ~{mtx-api/mtx-url}/-/cds/saas-provisioning/tenant/{tenantId}

parameters:
  service: saas-registry
  config:
    appUrls:
      getDependencies: ~{mtx-api/mtx-url}/-/cds/saas-provisioning/dependencies
      onSubscription: ~{mtx-api/mtx-url}/-/cds/saas-provisioning/tenant/{tenantId}

Miscellaneous Configuration ​

@sap/cds-mtx offers some additional configuration that you can also set in @sap/cds-mtxs.

HDI Container Configuration ​

In @sap/cds-mtx, you can configure the HDI container creation as follows:

"mtx": {
  "provisioning": {
      "lazymetadatacontainercreation": true,
      "container": {
          "provisioning_parameters": {
              "database_id": "<ID>"
          },
          "binding_parameters": {
            "key": "value"
          }
      },
      "metadatacontainer": {
        "provisioning_parameters": {
              "database_id": "<ID_META>"
          }
      }
  }
}

"mtx": {
  "provisioning": {
      "lazymetadatacontainercreation": true,
      "container": {
          "provisioning_parameters": {
              "database_id": "<ID>"
          },
          "binding_parameters": {
            "key": "value"
          }
      },
      "metadatacontainer": {
        "provisioning_parameters": {
              "database_id": "<ID_META>"
          }
      }
  }
}

In @sap/cds-mtxs, you can do the same configuration for the cds.xt.DeploymentService:

"requires": {
  "cds.xt.DeploymentService": {
    "lazyT0": true,
    "hdi": {
      "create": {
          "database_id": "<ID>"
      },
      "bind": {
        "key": "value"
      }
    },
    "for": {
      "t0": {
        "hdi": {
          "create": {
            "database_id": "<ID_META>"
          }
        }
      }
    }
  },
}

"requires": {
  "cds.xt.DeploymentService": {
    "lazyT0": true,
    "hdi": {
      "create": {
          "database_id": "<ID>"
      },
      "bind": {
        "key": "value"
      }
    },
    "for": {
      "t0": {
        "hdi": {
          "create": {
            "database_id": "<ID_META>"
          }
        }
      }
    }
  },
}

Extension Restrictions ​

This configuration allows to set what extensions are allowed.

With @sap/cds-mtx:

"mtx" : {
  "extension-allowlist": [
    {
        'for': ['my.bookshop.Authors', 'my.bookshop.Books'],
        'new-fields': 2
    },
    {
        'for': ['CatalogService']
    }
  ]
}

"mtx" : {
  "extension-allowlist": [
    {
        'for': ['my.bookshop.Authors', 'my.bookshop.Books'],
        'new-fields': 2
    },
    {
        'for': ['CatalogService']
    }
  ]
}

With @sap/cds-mtxs, the same configuration has moved to the cds.xt.ExtensibilityService configuration:

"requires": {
  "cds.xt.ExtensibilityService": {
    "extension-allowlist": [
      {
          'for': ['my.bookshop.Authors', 'my.bookshop.Books'],
          'new-fields': 2
      },
      {
          'for': ['CatalogService']
      }]
  }
}

"requires": {
  "cds.xt.ExtensibilityService": {
    "extension-allowlist": [
      {
          'for': ['my.bookshop.Authors', 'my.bookshop.Books'],
          'new-fields': 2
      },
      {
          'for': ['CatalogService']
      }]
  }
}

Migrate Tenant Content of Existing Applications ​

Depending on the MTX features that your existing application has used, you need to execute some steps to move your data to the persistence used by @sap/cds-mtxs.

Multitenancy Only ​

In case you only used the multitenancy features such as subscription/unsubscription, you just need to make the configuration changes described earlier.

{
    "subscriptionAppId": "...",
    "subscriptionAppName": "..." ,
    "subscribedTenantId": "...",
    ...
}

{
    "subscriptionAppId": "...",
    "subscriptionAppName": "..." ,
    "subscribedTenantId": "...",
    ...
}

See project configuration.

Saving Subscription Metadata ​

If your application needs access to the tenant list or tenant metadata, you need to update this data for @sap/cds-mtxs.

In order to copy the metadata from existing subscriptions to the new persistence of @sap/cds-mtxs, you need to run a migration script that comes with @sap/cds-mtxs.

Migration of Extensions ​

If your application supports extensibility, you also need to update the existing extensions for @sap/cds-mtxs. You can do this with the same migration script.

Run the Migration Script ​

The migration script is part of @sap/cds-mtxs. You can run it locally or during application deployment. Before running the script, you need to make the configuration changes mentioned earlier.

Run the Migration Script Locally ​

The script has to run in the (Node.js) application environment resulting from cds build --production to correctly simulate the execution in the deployment environment. For Node.js applications, this result is the gen/srv folder generated in the application root, for Java applications, this result is the gen folder of the new @sap/cds-mtxs sidecar (mtx/sidecar/gen).

It also needs access to the application bindings. That means, when running locally, it has to run in hybrid mode.

You also need to add the production profile to ensure that the models are resolved correctly.

Example:

cds migrate "*" --dry --profile hybrid,production --resolve-bindings

cds migrate "*" --dry --profile hybrid,production --resolve-bindings

Options ​

To run the migration for all or a set of tenants, you need to run:

cds migrate <tenant>[,<tenant>]|"*"

cds migrate <tenant>[,<tenant>]|"*"

The option --dry allows you to perform a dry run, without changing the database content.

Keep in mind that, depending on the number of tenants, the script requires some time to run. This is important when you consider to run it in combination with the application deployment.

If the migration was successful, tenants are marked as migrated. When running the migration a second time, these tenants are ignored. If you want to rerun the migration also for the already migrated tenants, you can do so by using parameter --force.

Save Existing Extension Projects ​

You can use the migration script to save the content of the subscribers' extension projects.

With parameter -d, you can specify a directory that is used by the script to store the existing, migrated extension projects.

cds migrate <tenant>[,<tenant>]|"*" -d <your directory>

cds migrate <tenant>[,<tenant>]|"*" -d <your directory>

To really access the saved extension projects, you need access to the file system, of course. So, the easiest way is to run the script locally for that.

Add the Migration Script as Cloud Foundry Task to mta.yaml ​

You can add the migration script as a hook to your Node.js server module (application or sidecar) in mta.yaml. For that, you can use the script cds-mtx-migrate that also comes with the @sap/cds-mtxs but doesn't require @sap/cds-dk to be installed. Example:

- name: bookshop-mt-sidecar
  type: nodejs
  path: mtx/sidecar
  ...
  hooks:
    - name: migrate-tenants
      type: task
      phases:
      #  - blue-green.application.before-start.idle
        - deploy.application.before-start
      parameters:
        name: migration
        memory: 512M
        disk-quota: 768M
        command: cds-mtx-migrate "*"

- name: bookshop-mt-sidecar
  type: nodejs
  path: mtx/sidecar
  ...
  hooks:
    - name: migrate-tenants
      type: task
      phases:
      #  - blue-green.application.before-start.idle
        - deploy.application.before-start
      parameters:
        name: migration
        memory: 512M
        disk-quota: 768M
        command: cds-mtx-migrate "*"

See also Module Hooks

Advanced: Separate Extensions Based on Extension File Names ​

The concept of extensions has slightly changed with @sap/cds-mtxs. Extensions sources are no longer stored in the backend. Instead, each extension gets a tag and the extension is stored as csn with the tag as key. When running the migration script, all extension files are compiled to one csn and are stored with a default tag: migrated.

You can change the default tag by passing your own tag using the --tag parameter:

cds migrate "*" -d migrated_projects --tag "mytag"

cds migrate "*" -d migrated_projects --tag "mytag"

In addition, you can separate your extensions into several csn-files with different tags. For example, if your original extension files follow a pattern, you can do so by passing parameter --tagRule with a regular expression.

Let's use the following extension project structure:

old-bookshop-ext/
├── db/
│   ├── extension_id_1.cds
│   └── extension_id_2.cds
│   └── order_ext_id_1.cds
│   └── order_ext_id_2.cds
├── srv/
└── package.json

old-bookshop-ext/
├── db/
│   ├── extension_id_1.cds
│   └── extension_id_2.cds
│   └── order_ext_id_1.cds
│   └── order_ext_id_2.cds
├── srv/
└── package.json

You can split your extensions as follows:

cds migrate "*" -d migrated_projects --tagRule "(?:ext_|extension_)(.*)\.cds"

cds migrate "*" -d migrated_projects --tagRule "(?:ext_|extension_)(.*)\.cds"

As a result, you get two extensions with tags id_1 and id_2. The tag is taken from the first captured group of the regular expression.

cds migrate "*" -d migrated_projects --tagRule "(?:ext_|extension_)(.*)\.cds" --dry

cds migrate "*" -d migrated_projects --tagRule "(?:ext_|extension_)(.*)\.cds" --dry

Check Migration Result ​

To verify the result of the migration script, check the tenant's content of the HDI container. You can use any database client that can access SAP HANA databases.

Check Content Using SAP HANA Database Explorer ​

To see the content of an HDI Container, you can add the tenant container to the SAP HANA Database Explorer.

You can find the migrated extensions in table CDS_XT_EXTENSIONS. The table contains:

• extensions parsed as csn strings in column csn
• key column tag

Migrated Extension Projects ​

As mentioned in Save Existing Extension Projects, you can store existing extension projects locally. We recommend to upload the projects to a source repository (e. g. github), because with @sap/cds-mtxs the content of extension projects is no longer stored in the tenant database. With that setup you can change and push the extension again later.

The content of extension projects is usually the property of the customer (subscriber). So, alternatively, the customer can download the extension projects himself and upload them to his own source repository.

Adapt for Streamlined MTX ​

As described in the extensibility guide, you usually start with an empty extension project and pull the base model of the application using cds pull.

When starting with a migrated extension project, you need to make some adaptations after running cds pull. Previously, extension projects were using the full set of CDS files whereas extension projects based on @sap/cds-mtxs are using a compiled index.csn of the base model. This affects the references in the extension sources of the migrated project. So these references need to be adapted.

Recommended steps:

• Delete the node_modules folder.
• Run cds pull to fetch the latest version of the base model as index.csn.
• Fix the references in your extension sources. All references must only contain _base.
  Example: using sap.capire.bookshop from '_base/db/schema'; must be replaced by using sap.capire.bookshop from '_base';
  You can see all broken references as error messages when using the CDS Editor.

Download of Migrated Extension Projects ​

As long as the metadata containers (TENANT-<id>-META) created by @sap/cds-mtx still exist, the customer extension projects can be downloaded using the CDS client. The user running the download command needs to have the scope cds.ExtensionDeveloper assigned:

cds extend <url> --download-migrated-projects [--tagRule <rule>] [--defaultTag <tag>]

cds extend <url> --download-migrated-projects [--tagRule <rule>] [--defaultTag <tag>]

The command downloads an archive named migrated_projects.tgz that contains the existing extensions that are ready to be used with @sap/cds-mtxs.
The parameter --tagRule <rule> is the regular expression that is used to split up the extension projects, the parameter --defaultTag <tag> is the tag that is to be used instead of the default tag migrated. See Separate Extensions Based on Extension File Names.

If the actual extension migration has been run with @sap/cds-mtxs version 1.13.0 or higher, you don't need to specify parameters --tagRule or --defaultTag. The extension project download uses the parameters that have been used for the actual migration.