# Data ingestion for the real world

> Connect your entire data stack, configure intelligent retries, and run custom backfills by table and endpoint.

Source: https://www.erathos.com/en/control
Em português: https://www.erathos.com/control

## An open API to integrate with the rest of your stack

Plug data ingestion into the rest of your workflow. Trigger jobs in Databricks when ingestion finishes, orchestrate with Airflow or Dagster, and run dbt transformations once the data is ready.

### Trigger jobs via API

Trigger and manage ingestion pipelines programmatically, straight from any external system.

### Orchestrate with your own tools

Plug ingestion into Airflow, Dagster, or any orchestrator that's already part of your stack.

### Coordinate with the rest of the pipeline

Chain ingestion together with downstream transformations and automations, as part of a bigger workflow.

## Configure the right retry for each API

Every API behaves differently. Some endpoints need aggressive retries. Others need larger backoff windows.

Erathos lets you define retry strategies at both the connection and job level, so pipelines recover from failures without overwhelming external systems.

### Custom retry strategies

Define how many times each job retries before it's flagged as an error.

### Configurable backoff

Adjust the wait time between attempts to make sure the run actually succeeds.

### Retry by connection or by job

Apply a default policy to the whole connection and override the logic for specific jobs when you need to.

## Reprocess historical data whenever you need to

Erathos lets you adjust cursors directly on job runs, making it easy to reprocess specific date ranges or recover from partial failures.

### Adjust cursors on runs

Change a job run's starting point directly on the run, without touching the pipeline configuration.

### Reprocess specific ranges

Reprocess from a specific date onward, without rerunning the whole pipeline.

### Detect deletions without a full refresh

Use backfill to compare against the source and mark removed records as soft-deleted, without reloading the whole table.

## Schedule pipelines with precision

Most ingestion tools schedule pipelines at the source level. But APIs rarely behave uniformly.

Erathos lets you schedule by endpoint or table, so each dataset syncs at the frequency it actually needs.

### Schedule by endpoint or table

Set the schedule at the level that makes sense, not just for the whole source.

### Run a specific endpoint whenever you want

Trigger a sync for a single endpoint at any time, without having to update the entire source.

### Fewer calls, fresher data

Avoid moving data that hasn't changed, cut API calls, and keep critical datasets always up to date.

## Frequently Asked Questions

### What does it mean to have control over data ingestion?

It means being able to decide how each pipeline behaves, not just check whether the data arrived. It involves triggering and orchestrating jobs via API, defining retry strategies per connection or job, reprocessing specific stretches of history without reloading everything, and scheduling by endpoint or table instead of by the whole source.

### Why does control over data ingestion matter?

Because ingestion without control turns into a black box. You only find out something went wrong once the data has already arrived incomplete or late at its destination. With control, you decide how each pipeline reacts to failures, fix history without depending on the tool's team, and adjust frequency to match each dataset's real needs.

### How does Erathos give me that control in practice?

With an open API to trigger and orchestrate jobs, retries configurable by connection or job, cursor-based backfill, and scheduling by endpoint — instead of leaving everything to a fixed default behavior.

### Does more control mean more configuration work?

Not necessarily. Erathos's defaults already cover most cases. You only step in when you want different behavior for a specific endpoint or job.

### How do I keep track of this day to day, without opening the configuration every time?

Every run is logged with what ran, when, and with what result — including retries, failures, and reprocessing. It's control and observability together.