Mapping framework

A Mapping algorithm allows you to state what fictitious values will replace the original data. It maps original data values to masked values pre-populated in a lookup table through the Continuous Compliance Engine user interface. There will be no collisions in the masked data because it always matches the same input to the same output. For example, David will always become Raj and Melissa will always become Jasmine. The algorithm checks whether an input has already been mapped; if so, the algorithm changes the data to its designated output.

The Mapping algorithm can be used on any set of values, of any length, but you must know how many values are being masked. To that end, provide at least the same amount of values as there are unique values being masking; though, more is acceptable. For example, if there are 10,000 unique values in the column being masked, you must give the Mapping algorithm at least 10,000 values.

The Mapping algorithm can be configured for mappings managed locally on the Continuous Compliance Engine or remotely on a user-managed PostgreSQL database. Remote mapping should be used for those who want to manage the storage allocated for mappings or share the same mappings from this algorithm across multiple DCT Compliance nodes. More information about remote mapping can be found in the Remote mapping page.

Configuration options

When creating an algorithm using the Mapping framework, configure the following options in the Configuration step.

Option	Description
Local Mapping Store	Stores mappings locally in DCT. When disabled, mappings are stored in a remote PostgreSQL database.
Ignore Characters	Characters that should be ignored when generating mappings. Multiple characters must be separated by commas.
Ignore Commas (,)	Specifies whether commas should be ignored when mapping values.

Mapping value requirements

The mapping table must contain at least as many output values as the number of unique values in the column being masked.

For example:

• If a column contains 10,000 unique values, you must provide at least 10,000 mapping outputs.

• Providing additional mapping outputs is allowed.

Mapping statistics

The algorithm overview page displays statistics about the mapping table.

Option	Description
Total Mappings	Total number of mapping outputs defined for the algorithm.
Available Mappings	Number of mapping outputs that have not yet been assigned to an input value.

These statistics help determine whether enough mapping values exist for future masking jobs.

Memory considerations

During a compliance job, mapping tables are loaded into memory.

Large mapping tables require more job memory.

For example:

• a mapping table with 2 GB of mappings requires more memory than one with 2 MB of mappings

Ensure that job memory limits are configured accordingly.

Using Mapping Algorithms with local mapping store

Before running a masking job that uses a mapping algorithm configured with a Local Mapping Store, additional configuration may be required depending on your deployment.

Kubernetes deployments

If DCT is deployed using Kubernetes and does not use an external database, the DCT mapping database service must be exposed as an external service so that the Compliance Node can access it.

By default:

• The mapping database runs as an internal Kubernetes service.

• The service type is ClusterIP.

• It runs on port 6432.

• Only DCT internal applications can access it.

To enable external access, the ClusterIP service must be exposed externally. The required steps vary depending on your Kubernetes distribution and organizational policies.

Contact your Kubernetes cluster administrator for guidance.

Example configurations

The following sections provide example approaches for exposing the mapping database service.

Using an ingress controller

If an ingress controller has already been configured, update the ingress configuration to enable TCP proxying for the mapping database port.

Example:

helm upgrade ingress-nginx ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace ingress-nginx \
--reuse-values \
--set tcp.6432="dct-services/database:6432"

This command preserves existing ingress configuration and adds a TCP mapping for port 6432.

Verify the configuration using:

kubectl get configmap tcp-services -n ingress-nginx -o yaml
kubectl get svc ingress-nginx-controller -n ingress-nginx

MicroK8s with built-in ingress addon

If DCT is deployed on MicroK8s, expose the mapping database service using the built-in ingress addon.

1. Create the TCP services ConfigMap

   kubectl apply -f - <<'EOF'
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: nginx-ingress-tcp-microk8s-conf
     namespace: ingress
   data:
     "6432": "dct-services/database:6432"
   EOF

2. Verify the controller configuration

   kubectl get daemonset nginx-ingress-microk8s-controller -n ingress -o yaml | grep -A2 tcp-services
   kubectl get daemonset nginx-ingress-microk8s-controller -n ingress -o yaml | grep -A3 6432

   If the configuration does not expose port 6432, update the daemonset accordingly.

3. Verify connectivity

   kubectl get pods -n ingress -w
   nc -zv localhost 6432

Configure global properties for external access

After exposing the mapping database service, configure the external connection settings.

1. In the DCT UI, go to Admin > Authentication > Global Properties.

2. Click Edit.

3. Configure the following properties.

   Property	Description
   Masking Mappings External DB Host	Hostname or IP address used by external applications to access the mapping database.
   Masking Mappings External DB Port	Port used to access the mapping database. Default value: 6432.

Deployment considerations:

• Virtual machine deployment – Use the public hostname or IP address of the VM. The default port typically works.

• Kubernetes deployment – Use the hostname or IP address where the mapping database service is exposed.

If a port other than 6432 is used when exposing the service, update the port property accordingly.

Troubleshooting

No mapping values

If no mapping values has been loaded, the job will fail and report the following error:

ERROR ... No mapping metadata found for algorithm '[MA_ALG]'.

Description:

• * MA_ALG is the name of the algorithm.

• * MA_COL is the column.

No free mappings available

If there are insufficient mappings available in the Mapping Algorithm when running a masking job, the job will fail and report an error:

WARNING-UNMASKED-DATA: RED_Mapping_Name failed to mask data at tcedCUST.CUST_NAME: Failed to map value. No free mappings are available. (4000000+ occurrences)

Description:

• * MA_ALG is the name of the algorithm.

• * MA_COL is the column

• * How many mappings are there

• * How many is missing

Create an algorithm using this framework

To create an algorithm using this framework, follow the steps in Create an algorithm and select Mapping during the Configuration step.