Solution Atlas
EverydayUser storyConsultative playbook

Our data-integration platform is on its third generation and still painful

The data team maintains a tangle of old jobs, custom Python scripts, and ageing integration tooling. A new integration takes weeks. Fabric Data Factory and Logic Apps offer a cleaner pattern, but adoption has to happen in stages without breaking what's running today.

Trigger
Integration backlog growing; old integration tooling is end-of-life.
Good outcome
New integrations go live in days rather than weeks. The old tooling is being retired in waves, and the customer can see, end-to-end, where every number on a report came from.
Discovery — signals and questions

Signals validating this story

  • ·SSIS estate aging; new integrations taking weeks
  • ·Heavy reliance on custom Python or Bash scripts
  • ·Tool fragmentation — three or more integration platforms
  • ·Pipeline lineage and observability informal
  • ·Integration tooling cost growing without commensurate value

Discovery questions

  1. 1.What is the current integration estate — SSIS, ADF, Logic Apps, Boomi, MuleSoft, custom?

    WhySizes the platform sprawl. Most customers have three or more.

    Listen for: “too many to count” · “each team picks their own” · “historical reasons”

  2. 2.How many integration patterns are live — batch, micro-batch, event-driven, streaming?

    WhySurfaces the platform-fit dimension. Different patterns need different tools.

  3. 3.What is the throughput and volume profile per integration?

    WhyDetermines whether Fabric Data Factory absorbs the workload or specialised tooling is needed.

  4. 4.Who owns each integration today, and what is the support model?

    WhyOften nobody owns the legacy SSIS jobs; they run because they ran yesterday.

  5. 5.What is your CI/CD posture for integrations?

    WhySource control + deployment is the maturity threshold. Surfaces engineering practice.

  6. 6.What is your lineage coverage today across the integration estate?

    WhyWithout lineage, every audit becomes archaeology.

Baseline architecture → Target architecture
Baseline architecture

Mix of SSIS, ad-hoc Python/Bash scripts, sometimes Boomi or MuleSoft. Integration code in fragmented repositories. Manual deployment. Pipeline monitoring uneven. Lineage informal. New integrations take weeks because the platform decision is re-litigated every time.

Typical concerns

  • ·SSIS estate aging; engineering interest declining
  • ·Multiple integration platforms with overlapping capability
  • ·No unified lineage across pipelines
  • ·Deployment manual; rollback painful
  • ·Cost of integration tooling growing without value

Capability gaps

  • ·Strategic integration platform with unified governance
  • ·Event-driven patterns for the right workloads
  • ·CI/CD for pipelines
  • ·Cross-platform lineage
  • ·Unified monitoring
Target architecture

Fabric Data Factory as the strategic integration platform — batch and micro-batch workloads, native OneLake integration, unified with the rest of Fabric. Logic Apps for event-based and SaaS connector scenarios. Azure Functions for serverless transformations. Source control + CI/CD for pipelines. Purview Data Governance provides cross-pipeline lineage. Unified monitoring via Azure Monitor.

Key capabilities

  • Strategic integration platform on Fabric
  • Event-driven integrations via Logic Apps
  • CI/CD for pipelines
  • Cross-platform lineage via Purview
  • Unified pipeline observability
Architecture decisions

  1. 1.Strategic platform — Fabric Data Factory vs standalone Azure Data Factory vs Synapse Pipelines

    Fabric Data Factory

    Fits whenFabric is the strategic data platform; OneLake-integrated integrations.

    Trade-offsTied to Fabric capacity.

    Azure Data Factory (standalone)

    Fits whenNon-Fabric estate; pure integration use case.

    Trade-offsLess native lineage; separate billing.

    Synapse Pipelines

    Fits whenAlready invested in Synapse; not yet migrated to Fabric.

    Trade-offsSame engine as ADF, but Synapse on a long deprecation curve.

    Default recommendationFabric Data Factory for new integrations and SSIS migration target. Standalone ADF only where Fabric is not yet in scope.

  2. 2.Migration approach — parallel run vs replace per workload

    Parallel run

    Fits whenCritical integrations where validation matters more than speed.

    Trade-offsTwo pipelines running for weeks; reconciliation cost.

    Replace per workload

    Fits whenNon-critical or low-risk integrations; cutover acceptable.

    Trade-offsCutover risk; rollback path needed.

    Default recommendationParallel run for tier-1; replace for tier-2 and tier-3.

  3. 3.Pattern selection — Data Factory (batch) vs Logic Apps (event) vs Functions (serverless transform)

    Data Factory for batch + micro-batch

    Fits whenScheduled or volume-driven workloads; ETL/ELT patterns.

    Trade-offsLatency bound to schedule.

    Logic Apps for event-driven + SaaS connectors

    Fits whenSaaS integrations (Salesforce, ServiceNow); event triggers; low-code surface.

    Trade-offsPer-execution cost at scale.

    Azure Functions for serverless transforms

    Fits whenCustom transformation logic; engineering team comfortable with code.

    Trade-offsOperational overhead for monitoring and lifecycle.

    Default recommendationData Factory as the default; Logic Apps for SaaS and event triggers; Functions for custom transforms only.

Low-risk trial — proof of value

8-week integration platform pilot

~8 weeks

Fabric Data Factory provisioned in the existing Fabric capacity. Five SSIS jobs migrated via parallel run. Two event-driven integrations stood up in Logic Apps. Source control + CI/CD configured for the new pipelines. Purview catalogues the new and reconciled pipelines with lineage.

Success criteria

  • Five SSIS jobs migrated with parity validated against parallel run
  • Two event-driven integrations in production via Logic Apps
  • CI/CD pipeline deploying integrations on commit
  • Purview lineage live for the migrated and new integrations

InvestmentConsumed against existing Fabric capacity. Purview Data Governance consumption ~€2–4k/month for the trial scope. Legacy SSIS continues running during parallel run.

Proof metrics

  • ·Time-to-deliver-new-integration reduced by 50%+
  • ·SSIS migration velocity measured per quarter
  • ·CI/CD adoption above 80% on new integrations
  • ·Purview lineage coverage above 70% on the migrated workloads

Recommended cards

The SKUs and capabilities most likely to be part of the solution, with the editorial rationale for each in the context of this story. Add the ones that fit your situation.

Why for this story

Fabric Data Factory is the strategic integration platform — integrated with OneLake, unified with the rest of Fabric, and the destination for SSIS modernisation. Capacity-based pricing replaces per-platform licensing.

Why for this story

Lineage across the integration platform. Surfaces "what feeds this report?" for every business consumer and produces the audit-defensible answer on demand.

Back to Data integration and pipelines