Watermark vs CDC in Azure Data Factory

Incremental loading moves only the data that has changed since a previous successful run. Watermarks and change data capture (CDC) support this goal through different mechanisms.

A watermark records a checkpoint from a source column, such as LastModifiedDate or an increasing identifier. The next pipeline run reads records beyond that checkpoint, copies them to the destination and advances the stored value after a successful load. This approach is simple when the source maintains a reliable change column.

Watermark pipelines need careful boundaries and recovery behaviour. Timestamp ties, late-arriving changes and failed runs can cause missed or repeated records unless the design accounts for them. Hard deletes usually require an additional mechanism because a deleted row is no longer available to a timestamp filter.

CDC reads changes tracked by a supported source system. Depending on the database and connector, those changes can include inserts, updates and deletes. A checkpoint such as a log sequence number identifies the last processed position. The destination must apply each change with the appropriate operation.

Choose a watermark when a dependable source column and a simpler pipeline meet the requirements. Consider CDC when delete tracking or more complete change history is important. Check source support, change retention, checkpoint recovery and destination merge behaviour before choosing the implementation.