# How to Deduplicate Data in BigQuery: MERGE, QUALIFY and ROW_NUMBER

> Find and remove duplicate rows in BigQuery with DISTINCT, ROW_NUMBER with QUALIFY and MERGE, and see what each method costs to run.

Source: https://www.erathos.com/en/blog/deduplicate-data-bigquery
Em português: https://www.erathos.com/blog/deduplicate-data-bigquery
Published: 2026-11-06
Category: Tutorials

![How to Deduplicate Data in BigQuery](https://cms-media.erathos.com/How to Deduplicate Data in BigQuery.png)

Your revenue dashboard shows more money than your finance team has booked. When you check the orders table in BigQuery, you see order 1002 twice, once with a status of “pending” and again as “paid”. You also see order 1001 twice, with the same values in every column, except for the time when the record was loaded. Any sum, count, or join against this table will be off.

Because the duplicates were introduced during data load, not in the source system, you will need to clean the data in the warehouse using SQL, as well as change the load process to prevent the same rows from getting loaded twice. In the following queries, I’ll use a small orders table to show what each method retains and what it drops.

## Why duplicates show up in BigQuery

Here are several loading patterns that lead to duplicates:

1. **A load fails and is then retried.** If a load fails mid-way, or takes too long and times out, the pipeline might try loading the data again. If some of the first load had already been written, the retry will cause the same data to be loaded twice. In streaming inserts, it’s [not possible to know if the insert succeeded](https://docs.cloud.google.com/bigquery/docs/streaming-data-into-bigquery) if there’s a network error, so a retry can create duplicates.
2. **Records change, and are loaded by appending.** Let’s say we’re loading orders. Each time the status of an order changes, we append a new row to the table. The original row is still there, and now there are multiple rows for the same order.
3. **An incremental load with overlapping windows.** An incremental load might select records that have changed since the previous load. If it’s using a filter like “greater than or equal to” the last value, or if it’s reloading the last few days’ worth of data just in case, some records at the edge will end up being loaded more than once.
4. **A full refresh load, appended instead of replaced.** If a job loads the entire table every time, it should be overwriting the target table. If it appends, then every run will create another copy of the entire table.
5. **No primary key in the source data.** Some API endpoints and log tables don’t have a column that uniquely identifies a record. In those cases, there’s no way to distinguish a duplicate from a non-duplicate, so they can accumulate unnoticed.

BigQuery won’t prevent any of these. Even if you declare a primary key on a table, [BigQuery only supports constraints that are not enforced](https://docs.cloud.google.com/bigquery/docs/information-schema-table-constraints). It will gladly allow you to insert another record with the same primary key.

Here’s an example of how the first and third causes can cause duplicate rows to be loaded for the same order:

## How do you find duplicate rows in BigQuery?

We can use `HAVING COUNT(*) > 1` on the business key of the table to find duplicate rows. These business keys have more than one row associated with them. The business key is the column that should be unique to a particular record, such as the order ID.

The examples in this guide are based on a table called `shop.raw_orders` with 5 rows. Order 1001 was loaded twice by a retry. Order 1002 was loaded once as pending and again as paid. Order 1003 is present only once.

`SELECT order_id, COUNT(*) AS copies`
`FROM `shop.raw_orders``
`GROUP BY order_id`
`HAVING COUNT(*) > 1`
`ORDER BY copies DESC;`

`order_id | copies`
`---------+-------`
`1001     | 2`
`1002     | 2`

Both orders look the same in this result, but they need different fixes. To tell them apart, count how many different `updated_at` values each key has:

`SELECT`
`  order_id,`
`  COUNT(*) AS copies,`
`  COUNT(DISTINCT updated_at) AS versions`
`FROM `shop.raw_orders``
`GROUP BY order_id`
`HAVING COUNT(*) > 1`
`ORDER BY order_id;`

`order_id | copies | versions`
`---------+--------+---------`
`1001     | 2      | 1`
`1002     | 2      | 2`

For higher `copies` than `versions`, some rows are copies of the exact same record, e.g. order 1001. For higher `versions` than one, some record changed, and each change became a row, e.g. order 1002. Most tables have both kinds of changes, and only the next two methods handle the second kind.

## Method 1: SELECT DISTINCT

`SELECT DISTINCT` returns exactly one row for each set of duplicate rows (where the duplicate rows are those with the same value in every column in the SELECT list). Therefore, it can only be used on exact duplicates.

On the sample table, `SELECT DISTINCT *` returns all five rows. The two copies of order 1001 differ in `_loaded_at`, the column the loader fills with the load time, so DISTINCT treats them as different rows. Leaving that column out fixes the exact copy:

`SELECT DISTINCT * EXCEPT (_loaded_at)`
`FROM `shop.raw_orders`;`

`order_id | customer_id | status  | amount | updated_at`
`---------+-------------+---------+--------+------------------------`
`1001     | 101         | paid    | 120.00 | 2026-10-19 18:00:00 UTC`
`1002     | 102         | paid    | 80.00  | 2026-10-20 09:00:00 UTC`
`1002     | 102         | pending | 80.00  | 2026-10-19 10:00:00 UTC`
`1003     | 103         | shipped | 45.50  | 2026-10-18 08:00:00 UTC`

Order 1001 was reduced to one row, but 1002 still had two, because the rows have different values for status and updated\_at. DISTINCT doesn't know which version is the newer one. Therefore, tables receiving updates must use some method of explicitly selecting one row per key.

## Method 2: ROW\_NUMBER() with QUALIFY

This is the query to use when a table has several versions of the same record and you want the newest one:

`SELECT *`
`FROM `shop.raw_orders``
`QUALIFY ROW_NUMBER() OVER (`
`  PARTITION BY order_id`
`  ORDER BY updated_at DESC, _loaded_at DESC`
`) = 1;`

`order_id | customer_id | status  | amount | updated_at              | _loaded_at`
`---------+-------------+---------+--------+-------------------------+------------------------`
`1001     | 101         | paid    | 120.00 | 2026-10-19 18:00:00 UTC | 2026-10-20 03:05:00 UTC`
`1002     | 102         | paid    | 80.00  | 2026-10-20 09:00:00 UTC | 2026-10-20 12:00:00 UTC`
`1003     | 103         | shipped | 45.50  | 2026-10-18 08:00:00 UTC | 2026-10-18 12:00:00 UTC`

In this result, we see 3 orders with 1 row per order. Order 1002 is marked "paid". Here’s what’s going on:

- **ROW\_NUMBER()** assigns numbers to rows starting with 1. It’s a [window function](https://cloud.google.com/bigquery/docs/reference/standard-sql/window-function-calls) which means it operates on a set of rows rather than a single row.
- **PARTITION BY order\_id** resets the numbering for each order.
- **ORDER BY updated\_at DESC** orders the versions for each order such that the newest version comes first.
- **QUALIFY … = 1** filters down to the latest version of each order. [QUALIFY filters based on window functions](https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax) the way WHERE filters based on plain columns.

If you don’t have QUALIFY, you can compute row number as a column in a subquery then filter `WHERE row_num = 1` in the outer query. It does the same thing as the example above but with one more layer of nesting.

The second column in the ORDER BY breaks the tie. Since both copies of order 1001 have the same `updated_at`, the tie-breaker is required to make the ORDER BY unique per row. If the rows are identical on all ORDER BY columns, then BigQuery is free to return the rows in any order and could return a different row on a subsequent run. This is fine for exact copies. However, if two different versions of the same order had the same `updated_at`, then the latest row could change from run to run. Adding the load time (or any other column that is unique per row) fixes this problem.

One limitation of the key is that floating point types [can’t be used in the PARTITION BY expression](https://cloud.google.com/bigquery/docs/reference/standard-sql/window-function-calls). If your key is stored as FLOAT64, then cast it to a string or integer first.

## Method 3: MERGE

The first two approaches, DISTINCT and QUALIFY, clean data as it’s read. MERGE is used to keep a clean table up-to-date as new batches come in, without ever having duplicates in the table. The pipeline loads each new batch into a staging table, and then runs a MERGE statement to update existing orders and insert new orders. This is commonly referred to as an upsert (update or insert).

The pipeline below shows how each batch of data is processed.

In the example, the clean table `shop.orders` has three orders, with 1002 still at "pending." A new batch arrives in `shop.orders_batch`. It contains order 1002 as "paid" twice, because a retry loaded it twice. It also has a new order 1004, and order 1003 with no change:

`MERGE `shop.orders` AS t`
`USING (`
`  SELECT *`
`  FROM `shop.orders_batch``
`  QUALIFY ROW_NUMBER() OVER (`
`    PARTITION BY order_id`
`    ORDER BY updated_at DESC, _loaded_at DESC`
`  ) = 1`
`) AS s`
`ON t.order_id = s.order_id`
`WHEN MATCHED AND s.updated_at > t.updated_at THEN`
`  UPDATE SET`
`    status = s.status,`
`    amount = s.amount,`
`    updated_at = s.updated_at,`
`    _loaded_at = s._loaded_at`
`WHEN NOT MATCHED THEN INSERT ROW;`

After the MERGE, the clean table has one row per order:

`order_id | customer_id | status  | amount | updated_at              | _loaded_at`
`---------+-------------+---------+--------+-------------------------+------------------------`
`1001     | 101         | paid    | 120.00 | 2026-10-19 18:00:00 UTC | 2026-10-20 03:05:00 UTC`
`1002     | 102         | paid    | 80.00  | 2026-10-20 09:00:00 UTC | 2026-10-20 12:05:00 UTC`
`1003     | 103         | shipped | 45.50  | 2026-10-18 08:00:00 UTC | 2026-10-18 12:00:00 UTC`
`1004     | 104         | pending | 60.00  | 2026-10-21 07:00:00 UTC | 2026-10-21 08:00:00 UTC`

Each clause performs its own specific function:

- `ON t.order_id = s.order_id`: For each row in the batch, join to the row in the clean table with the same key.
- `WHEN MATCHED AND s.updated_at > t.updated_at`: Only update the row if we’ve received a newer version of the order in our batch. Order 1003 is already present in the clean table and didn’t change in our batch, so the MERGE leaves it alone.
- `WHEN NOT MATCHED THEN INSERT ROW`: Insert any orders we’ve never seen before (order 1004).

Note that the QUALIFY inside the USING clause is crucial. MERGE statements with an UPDATE clause cannot have two rows in the source table match the same row in the target table; if they do, BigQuery will error out with [`UPDATE/MERGE must match at most one source row for each target row`](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax). The duplicate 1002 rows in the batch would run afoul of this, so we first deduplicate the batch.

One tricky case is NULL keys. Since `NULL = NULL` is not true in SQL, a row in the source table with a NULL key will never match the target table and get inserted again on every run. If your key can be NULL, you should either match using `t.key = s.key OR (t.key IS NULL AND s.key IS NULL)`, or filter out such rows from the batch.

## What each method costs on on-demand pricing

The pricing model is on-demand, and charges per byte of query reads (not rows returned). At the time of writing, BigQuery charges [$6.25 per TiB processed, after the first free TiB each month](https://cloud.google.com/bigquery/pricing). It only counts the columns that are read, and doesn't get discounted by a LIMIT. There is a 10 MB minimum for each table in a query.

This means that both the DISTINCT and the QUALIFY approach will be charged the same amount of bytes for the same columns read from the same table, because even though the window function takes more work, it doesn't take more bytes to read.

By contrast, the MERGE approach will be charged for both [the bytes read plus the size of the target table](https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax) if it has an UPDATE or DELETE clause (though if it only has INSERT clauses, then it will only be charged for the bytes read).

Let's calculate the cost of one example scenario. Let's say we have a raw table of size 500 GiB and a clean table of size 200 GiB. Each daily batch of data is 2 GiB, and the size of the key and `updated_at` columns of the clean table is 8 GiB. Remember that 1 TiB = 1,024 GiB.

`Rebuild the clean table from raw every day (DISTINCT or QUALIFY)`
`  500 GiB / 1,024 = 0.488 TiB x $6.25 = $3.05 per run`
`  30 runs = $91.55 per month`

`MERGE each daily batch (UPDATE + INSERT)`
`  read:   2 GiB batch + 8 GiB target columns = 10 GiB`
`  target: 200 GiB`
`  210 GiB / 1,024 = 0.205 TiB x $6.25 = $1.28 per run`
`  30 runs = $38.45 per month`

`MERGE each daily batch (INSERT only)`
`  10 GiB / 1,024 = 0.0098 TiB x $6.25 = $0.06 per run`

The rebuild will pay the cost of processing the whole raw table every time, and the raw table will continue to grow over time, while MERGE processes the clean table, which only grows with new records. If data does not change after it has been written (e.g. events, logs), then an insert-only MERGE is the cheapest way to update it.

Method

Use it for

What one run bills

Example cost per run

SELECT DISTINCT

Exact copies only

Columns read from the whole table

$3.05

ROW\_NUMBER() + QUALIFY

Several versions of one record

Same columns, same bytes as DISTINCT

$3.05

MERGE (update + insert)

Keeping a clean table current

Batch + target columns read + whole target table

$1.28

MERGE (insert only)

Records that never change

Batch + target columns read

$0.06

For small tables of a few GiB, a full rebuild with QUALIFY will be cheaper than the MERGE approach, and more straightforward to set up. But as soon as the raw table is much larger than the new batch, the MERGE approach will become more cost-effective.

Here's the summary of all three methods:

## Cutting cost with partitioning and clustering

Note that the MERGE above would still pay for the whole 200 GiB target on each run. [Partitioning](https://docs.cloud.google.com/bigquery/docs/partitioned-tables) the clean table by date would split the table into chunks for each day, while [clustering](https://cloud.google.com/bigquery/docs/clustered-tables) by the key would sort the data in each of the chunks so that a filter or join on the key can skip blocks without it.

Here is a clean table partitioned by order date and clustered by order ID:

`CREATE TABLE `shop.orders_p` (`
`  order_id INT64,`
`  order_date DATE,`
`  status STRING,`
`  amount NUMERIC,`
`  updated_at TIMESTAMP`
`)`
`PARTITION BY order_date`
`CLUSTER BY order_id;`

A partitioned table only helps when the MERGE tells BigQuery which partitions to touch. Adding a filter on the partition column to the ON clause [limits which partitions BigQuery scans](https://docs.cloud.google.com/bigquery/docs/using-dml-with-partitioned-tables):

`MERGE `shop.orders_p` AS t`
`USING `shop.orders_p_batch` AS s`
`ON t.order_id = s.order_id`
`  AND t.order_date >= DATE '2026-10-01'`
`WHEN MATCHED AND s.updated_at > t.updated_at THEN`
`  UPDATE SET`
`    order_date = s.order_date,`
`    status = s.status,`
`    amount = s.amount,`
`    updated_at = s.updated_at`
`WHEN NOT MATCHED THEN INSERT ROW;`

The bill now covers only the partitions the filter keeps. Say the 200 GiB clean table holds 1,000 days of orders spread evenly, so each day is 0.2 GiB, and the filter keeps the last 30 days:

`target partitions: 30 days x 0.2 GiB = 6 GiB`
`read: 2 GiB batch + 0.24 GiB target columns in those days = 2.24 GiB`
`8.24 GiB / 1,024 = 0.008 TiB x $6.25 = $0.05 per run`
`30 runs = $1.51 per month (down from $38.45)`

That filter has a cost of its own. Order 2001 was placed in September and paid in October. Its update arrives in the batch, but the September partition is outside the filter, so the MERGE can't find the old row. It falls through to WHEN NOT MATCHED and gets inserted as a second row:

`order_id | order_date | status  | amount | updated_at`
`---------+------------+---------+--------+------------------------`
`1001     | 2026-10-19 | paid    | 120.00 | 2026-10-19 18:00:00 UTC`
`1002     | 2026-10-19 | paid    | 80.00  | 2026-10-21 08:00:00 UTC`
`2001     | 2026-09-15 | pending | 30.00  | 2026-09-15 10:00:00 UTC`
`2001     | 2026-09-15 | paid    | 30.00  | 2026-10-21 09:00:00 UTC`

The fix is to take the window from the batch itself, so it always reaches back to the oldest order the batch touches:

`DECLARE min_date DATE DEFAULT (`
`  SELECT MIN(order_date)`
`  FROM `shop.orders_p_batch``
`);`

`MERGE `shop.orders_p` AS t`
`USING `shop.orders_p_batch` AS s`
`ON t.order_id = s.order_id`
`  AND t.order_date >= min_date`
`WHEN MATCHED AND s.updated_at > t.updated_at THEN`
`  UPDATE SET`
`    order_date = s.order_date,`
`    status = s.status,`
`    amount = s.amount,`
`    updated_at = s.updated_at`
`WHEN NOT MATCHED THEN INSERT ROW;`

This time order 2001 will match, and the table will contain one row for each order. When first running the query, check the processed bytes in the job details to verify that only a small amount of data was scanned. This works because the order date will never change for a given order, but a column that can change, such as updated\_at, is not a good choice of partition column for a MERGE target.

A few more things to watch out for:

- BigQuery will [scan the entire target table](https://docs.cloud.google.com/bigquery/docs/using-dml-with-partitioned-tables) if you use WHEN NOT MATCHED BY SOURCE, even if the ON clause contains a filter. Since this clause deletes rows from the target if they are not present in the batch, unless you need to do this, then don't include this clause.
- BigQuery cannot tell how many blocks the query will skip [until the query is executed](https://cloud.google.com/bigquery/docs/clustered-tables). Therefore, the estimated price before running the query will be higher than the final billed price.

The cost can be reduced even further, so please refer to our guide to [BigQuery cost optimization](https://www.erathos.com/en/blog/bigquery-cost-optimization?utm_source=blog&utm_medium=organic&utm_content=bydefault&utm_campaign=deduplicate-data-bigquery) for more details.

## Keep raw data as is and build a deduplicated layer

Deleting duplicates from the raw table might look more tidy, but it means discarding the only version of the data that came from the source. In case of any bug in the dedup logic, there is nothing left to rebuild from. Keeping the raw table exactly as it was loaded, and building the cleaned up version on top of it, is more safe. The data would then pass through two levels:

This is the bronze and silver split from the [medallion architecture](https://www.erathos.com/en/blog/medallion-architecture-complete-guide?utm_source=blog&utm_medium=organic&utm_content=bydefault&utm_campaign=deduplicate-data-bigquery). The bronze layer stores data as it arrives from the source, duplicates included. Deduplication is one of the first steps in the silver layer. The QUALIFY query from Method 2 becomes the definition of the clean layer:

`CREATE OR REPLACE VIEW `shop.orders_clean` AS`
`SELECT *`
`FROM `shop.raw_orders``
`QUALIFY ROW_NUMBER() OVER (`
`  PARTITION BY order_id`
`  ORDER BY updated_at DESC, _loaded_at DESC`
`) = 1;`

To store the result instead, change `CREATE OR REPLACE VIEW` to `CREATE OR REPLACE TABLE` and run it on a schedule. The three options cost money in different places:

Clean layer

Freshness

Who pays the scan

View

Always current

Every query that reads the view rescans the raw table

Table rebuilt on a schedule

As of the last rebuild

The rebuild, once per run

Table kept current with MERGE

As of the last batch

The MERGE, once per batch

A view costs nothing to create. Instead, it accrues more and more costs as it gets read. So, if we take the sizes from the cost section above, then a dashboard that reads the view 20 times a day with SELECT \* ends up scanning the 500 GiB raw table 20 times, costing 20 x $3.05 = $61.04 a day. On the other hand, the table costs $3.05 a day if we rebuild it once a day. When it’s read, we’re only scanning the clean table.

A materialized view seems like it sits in the middle, but [materialized views don’t support window functions](https://docs.cloud.google.com/bigquery/docs/materialized-views-create), so we can’t use the QUALIFY with ROW\_NUMBER pattern there. Non-incremental materialized views, on the other hand, support window functions, but [such views are refreshed in full](https://docs.cloud.google.com/bigquery/docs/materialized-views-create), taking us back to the cost of a rebuild.

Sometimes you don’t want the latest version of a record; you want to see all versions of it, perhaps so that you can see how an order transitioned from being pending to being paid. In this case, we don’t delete the versions, we just model them as a [slowly changing dimension type 2](https://www.erathos.com/en/blog/slowly-changing-dimension-type-2-scd2?utm_source=blog&utm_medium=organic&utm_content=bydefault&utm_campaign=deduplicate-data-bigquery) table.

## Preventing duplicates upstream

To clean up the copies, we use a different strategy in each of the above queries. The more cost-effective approach is to make sure we don’t create them in the first place. This can be achieved with the following three habits:

1. **Make loads idempotent:** If a load is idempotent, then it produces the same result regardless of whether it’s run once or three times. This is the case with MERGE operations on the key, where a second run would find the rows already inserted. For streaming, the insert ID used in older streaming insert queries only provides [best-effort deduplication for up to one minute](https://docs.cloud.google.com/bigquery/docs/streaming-data-into-bigquery). Exactly-once delivery is instead provided by the Storage Write API.
2. **Agree on a primary key at the source:** All the methods described here rely on some column or set of columns uniquely identifying a record. Tables and API endpoints without such a column are the most difficult to clean up, so find or create such a key before the data is written.
3. **Pick the update type per table:** The choice of how the table is synced determines whether updates create new rows or replace old ones.

In Erathos, our platform for loading data from databases and APIs into warehouses like BigQuery, the update type is a setting on each table, so one connection can use different types for different tables. These are the [sync types the docs describe](https://docs.erathos.com/platform/connections/sync-types):

Update type

What happens in the destination

Fits

Full Refresh

Reads the whole source table and overwrites all values in the destination

Small tables, or tables with no key or cursor

Partial Overwrite

Inserts new records, and updated records overwrite the outdated ones

Tables where you only need the current state

Partial Append

Inserts every record, so updated records appear more than once

Keeping every change, cleaned later with QUALIFY

Partial Versioned

Like Partial Append, with a flag that marks the most recent version

Keeping history, as in an SCD2 table

The partial types load only records whose cursor column (e.g. `updated_at`) is greater than the last value loaded. They need a primary key and a date/datetime cursor column. With the Partial Append and Partial Versioned types, which load more than one row per record on purpose, the QUALIFY view from the previous section is the clean layer. Erathos describes the three partial types as follows:

A cursor can miss changes, like deleted rows, so Erathos also offers change data capture (CDC) on the Pro plan. CDC reads changes from the database’s log, not from a cursor column. The tradeoffs are described in our guide to [cursor-based sync vs. change data capture](https://www.erathos.com/en/blog/cursor-based-sync-vs-change-data-capture?utm_source=blog&utm_medium=organic&utm_content=bydefault&utm_campaign=deduplicate-data-bigquery). For partial syncs, there’s an “Enable Full On” setting that schedules a full refresh from time to time to fix any drift between source and destination.

Retries can be customized as well. With a [custom retry strategy](https://docs.erathos.com/platform/connections/custom-retry-strategy), you can choose the number of attempts before Erathos shows an error and sends an alert, as well as the wait time between attempts. You can configure it on a connection level, so it applies to all its jobs, or on a single job, in which case it overrides the connection setting. Custom retries are available on the Pro and Enterprise plans.

## FAQ

**Does BigQuery enforce primary keys?** No. Primary keys can be declared, but BigQuery [only supports unenforced constraints](https://docs.cloud.google.com/bigquery/docs/information-schema-table-constraints). BigQuery will accept two rows with the same key.

**Is QUALIFY cheaper than a subquery with ROW\_NUMBER?** No. Both read the same columns. Both bill the same number of bytes and return the same rows. QUALIFY is shorter to write.

**Does adding LIMIT make a dedup query cheaper?** No. On-demand pricing bills the columns a query reads, and [a LIMIT doesn’t change that](https://cloud.google.com/bigquery/pricing). Selecting fewer columns or filtering on a partition column does.

**Why does my MERGE fail with "must match at most one source row for each target row"?** Two or more rows in the source have the same key. The MERGE has an UPDATE clause. Deduplicate the source using QUALIFY inside the USING clause. See Method 3.

**Can I put QUALIFY in a materialized view?** No. Standard materialized views [don’t support window functions](https://docs.cloud.google.com/bigquery/docs/materialized-views-create). Use a regular view instead. Or use a table rebuilt on a schedule.

**Can I run MERGE in the BigQuery sandbox?** No. The BigQuery sandbox does [not have access to DML statements](https://docs.erathos.com/destinations/what-is-a-destination/bigquery), and MERGE is a DML statement. Enabling billing on the Google Cloud project takes it out of sandbox mode.

**Should I delete duplicates from the raw table?** No. Keeping the raw table as loaded and building a clean view or table on top is safer. If the cleanup logic is wrong, you can fix it and rebuild from the raw data.

## Try it on your own data

Duplicates are much easier to handle when each table is loaded the right way from the start. In Erathos, you choose the update type and the retry settings for every table and job, and BigQuery is one of the destinations. You can [try Erathos free for 14 days](https://app.erathos.com/signup?utm_source=blog&utm_medium=organic&utm_content=bydefault&utm_campaign=deduplicate-data-bigquery) with access to the Pro plan.
