Piotr MuchaPrincipal Software Engineer FairyDeck EN / PL Discuss your system
Menu

Production case · TecAlliance data

Architecture delivered to production

Automating catalog updates for nearly 10 million automotive products.

The data flow removed recurring developer work and made each published product delta available to the business automatically.

Nearly 10M
standardized automotive products
Thousands → tens of thousands / week
changes in a weekly delta

01

Turn catalog maintenance into a repeatable data product.

Product data arriving from suppliers was often incomplete or inconsistent. Downloading, formatting and loading the external catalog took days of developer work, so updates happened only every few months.

The new architecture uses standardized TecAlliance data and automatically processes weekly deltas when they become available. It keeps the catalog materially more current without claiming real-time source publication.

02

A data-quality problem sustained by a manual update cycle.

Initial state

Incomplete supplier data and developer-dependent imports

Maintaining a consistent automotive catalog required repeated manual preparation. Because a full update consumed several days, the organization refreshed the data only once every few months.

Constraints

Large volumes with a weekly external change cadence

  • The standardized catalog contains nearly 10 million products.
  • Weekly deltas range from several thousand to tens of thousands of changes.
  • A full update and incremental updates require different processing paths.
  • The architecture had to be implementable and operable by the delivery team.

Data strategy

Automate after publication instead of promising real-time source data

The system reacts automatically when TecAlliance makes changes available. This removes internal delay while preserving the real weekly cadence of the external source.

My contribution

Architecture ownership with an explicit implementation boundary.

I designed the solution and the end-to-end data flow. Other developers implemented the production system.

Catalog ingestion architecture
I defined how full data and deltas enter the platform, are processed and become available to downstream systems.
Standardized product model
The design treated TecAlliance as the source of consistent automotive reference data instead of preserving supplier-specific shapes.
Implementation by the team
Developers implemented the architecture in the production environment using EKS, ECR and RDS components.

This case claims architecture and data-flow ownership, not direct implementation by Piotr. It does not imply TecAlliance endorsement.

Key decisions

Separate source cadence from internal processing delay.

The goal was to process every available change automatically and consistently, without overstating how frequently the external source publishes data.

  1. 01

    Full load and delta paths

    Large complete datasets and weekly incremental changes follow processing suited to their size and purpose.

  2. 02

    One standardized catalog

    Downstream systems consume a consistent model instead of repeating supplier-data interpretation.

  3. 03

    Automatic reaction to published changes

    The pipeline starts when a new delta becomes available and removes the previous internal waiting period.

Delivery model

From data contract to an operable production pipeline.

  1. 01

    Define the canonical product input

    Identify the standardized reference data and the boundaries between full loads, deltas and downstream consumption.

  2. 02

    Design automated processing

    Specify triggers, storage, processing stages, failure visibility and the production infrastructure boundary.

  3. 03

    Implement and operate as a team

    The delivery team implemented the architecture and moved recurring weekly updates into normal operation.

Verified production outcome

The catalog updates when external changes become available.

Nearly 10M
covered by the standardized catalog
Thousands → tens of thousands / week
processed after a weekly delta is published
Developer-days → automatic processing
developer-dependent, multi-day work replaced by automatic processing

Automatic processing begins after a weekly source delta is available. The case does not claim real-time source updates.

What this proves

Large-scale data integration designed around business freshness.

The project demonstrates how architecture can remove recurring manual work, improve catalog consistency and remain honest about the boundary between design ownership and team implementation.

Modernizing a critical ERP without stopping sales, purchasing or warehouse operations.

A multi-channel automotive parts wholesaler relied on several undocumented, tightly coupled legacy systems. Instead of rewriting everything at once, I moved complete business processes into auditable domains while keeping sales, purchasing and warehouse operations online.

Normalizing European PSD2 bank integrations for an Account-to-Account platform handling hundreds of thousands of transactions each month.

European banks exposed different APIs, statuses and failure semantics. I implemented integrations that mapped these contracts into one Account-to-Account payment flow with Mutual TLS, certificate handling and explicit behavior for retries, duplicates and ambiguous states.

Your data flow

Does a recurring import still depend on developer intervention?

We can define the source contract, update cadence and operating model for an integration that the team can trust.

Discuss a data integration architecture

Back to selected work