Performance logging for masking jobs provides structured, repeatable metrics that help you understand how a masking job behaves over time. From version 2026.3, the existing ThroughputLogger is enhanced and renamed to PerfLogger. PerfLogger records performance data at defined phases of a job so that you can evaluate startup behaviour, steady‑state throughput, and overall job performance.

You can use PerfLogger output to validate expected throughput, detect performance regressions, and troubleshoot slowdowns or hangs in long‑running jobs.

How performance logging works

PerfLogger operates in hybrid mode, which logs performance data using different strategies as the job progresses. Each phase provides insight into a specific aspect of job execution.

Logging phases in hybrid mode

PerfLogger records performance data in three phases:

  • Startup phase (row‑based logging)

    Logs performance frequently at the beginning of the job to show how quickly rows are ingested and how fast the internal buffer fills.

  • Steady‑state phase (time‑based logging)

    Logs performance at fixed time intervals while input and update (or insert) steps operate in lock‑step.

  • Completion phase (final summary)

    Logs a single summary entry that represents total job performance.

What you can use PerfLogger for

PerfLogger metrics help you:

  • Validate expected overall runtime performance (RPM).

  • Validate input performance during job startup.

  • Detect slowdowns, stalls, or hangs in long‑running jobs.

  • Identify unusually large input or masked output rows.

  • Compare performance across environments or job executions.

Startup phase logging (row‑based)

What happens during the startup phase

During startup, PerfLogger logs performance after a fixed number of rows are masked. This phase focuses on input throughput and buffer behaviour.

  • Logging occurs every 10,000 rows masked.

  • The row count reflects the internal buffer size.

  • The initial spike typically represents peak input RPM.

Example startup log entry

Initializing PerfLogger with hybrid mode: 10000 rows for 10 iterations, then 30000 ms intervals
...
PerfLogger:time ; 214 ms ; 10000 rows ; 0.48 MB ; 2803738 RPM ; 48 BPR ; 2.22 MB/sec

Startup phase fields

  • ms – Logging interval.

  • rows – Rows processed (fixed at 10,000).

  • MB – Data size processed during the interval.

  • RPM – Rows per minute (primary performance indicator).

  • BPR – Average bytes per row.

  • Large values (kilobyte range) often cause performance degradation.

  • MB/sec – Throughput during the interval (informational only).

Steady‑state logging (time‑based)

What happens during steady‑state logging

After the job processes 100,000 rows, PerfLogger switches to time‑based logging.

  • Logging occurs every 30 seconds.

  • Each entry reports performance for the previous 30‑second interval.

  • Input and update (or insert) steps run in lock‑step, with new input retrieved only after a batch update completes.

This phase helps you observe sustained runtime behaviour and identify gradual performance changes.

Example steady‑state log entry

PerfLogger:time ; 30000 ms ; 492659 rows ; 23.81 MB ; 985318 RPM ; 48 BPR ; 0.79 MB/sec

Steady‑state fields

  • ms – Logging interval (fixed at 30 seconds).

  • rows – Rows processed during the interval.

  • MB, RPM, BPR, MB/sec – Same definitions as in startup logging.

Completion logging (final summary)

What happens at job completion

When the masking job finishes, PerfLogger writes a single summary entry that represents total job performance.

Example final summary entry

PerfLogger:total ; 125092 ms ; 2000000 rows ; 96.66 MB ; 959293 RPM ; 48 BPR ; 0.77 MB/sec

Final summary fields

  • ms – Total runtime.

  • rows – Total rows processed.

  • MB – Total data size processed.

  • RPM – Average rows per minute for the entire job.

  • BPR – Average bytes per row.

  • MB/sec – Average throughput (informational only).

Troubleshoot performance issues using PerfLogger

Use PerfLogger metrics to identify common performance patterns. Focus primarily on RPM, not MB/sec.

RPM drops significantly

A sharp or sustained RPM drop—especially to zero—often indicates a target‑side bottleneck, assuming BPR has not increased significantly. Common causes include:

  • Transaction log pressure.

  • Database growth or auto‑resize events.

  • Large or numerous indexes being updated.

If RPM does not recover, the job may be stalled or hung.

RPM is low but MB/sec is high

When masked rows are large (approximately 1 KB per row or more), RPM typically decreases because more data moves through:

  • Input processing.

  • Masking logic.

  • Update or insert operations.

  • Network and storage layers.

For this reason:

  • MB/sec is not a reliable performance indicator.

  • RPM is the primary metric to evaluate.

Performance is below previous baselines

If performance is significantly worse than earlier runs with similar data, consider the following factors:

  • Multiple masking jobs running concurrently.

  • Schema changes, such as new indexes, constraints, or triggers.

  • Increased row size (compare BPR values).

 

Last updated
Apr 29, 2026