შინაარსზე გადასვლა
operations21 ოქტომბერი, 2026 · 11 min read

Why USDT deposits get stuck: the failure classes we found in production

Silent minimums, sweeps marked done without a receipt, an empty gas tank, scanners that die quietly: deposit failures we found running EX5, and the alert for each.

Kostiantyn Boiko, Blockchain Engineer

English original

“I sent USDT and it’s not in my account” is the ticket every exchange support desk knows. Between July and October 2026 we classified every stuck deposit we found in the EX5 exchange and the EX5 Wallet, including the ones no customer had reported yet.

Most of them were not stuck in the way the ticket implied. A deposit passes three gates, and each one fails differently:

  1. Seen. Did the scanner notice the transfer at all?
  2. Credited. Did the customer’s balance go up?
  3. Swept. Did the coins move from the customer’s deposit address to the hot wallet?

The customer only cares about the second. Operations has to watch all three, because a failure at the third gate can stay invisible for weeks.

Seen, and then ignored: the silent minimum

Every network has a minimum deposit. Below it, our scanner did exactly what the code said: it returned early and wrote one line to a log. No database row, no pending entry, nothing a screen or an alert could ever see. The coins sat on the deposit address, and the customer and the support agent were both looking at an empty history.

The case that taught us this was a customer who split one deposit into two transfers. Each part was under the 15 USDT minimum and the total was over it, so neither part was recorded. Those funds sat uncredited for about 50 days.

What we changed:

  • We lowered the TRC-20 USDT minimum from 15 to 10 USDT on both stacks and credited the one deposit that had really been missed.
  • Lowering a threshold does not reach back in time. To pick up an old deposit you move the scanner’s height back past it (we went back about 150 blocks) or credit it by hand.
  • A treasury watcher now flags any deposit address that holds funds with no open deposit record. The rule matters: “no record at all” would miss an address that had an earlier deposit.

Before crediting anything by hand, check that it is not already credited. Most of the addresses flagged the first time were not missing deposits at all: two had been credited by an earlier balance migration and one held an internal test transfer. An internal review of our own recovery plan caught a recommendation that would have credited two customers twice.

The rule we would give anyone: publish the minimum per network, show it on the deposit screen next to the address, and monitor balances that have no record behind them.

Never seen: detection that fails without an error

The worst detection failures produce no error at all. Every component reports healthy, and deposits simply never appear.

  • A configuration field in the wrong place. On six ERC-20 tokens, the token contract address had ended up in the wrong field of the currency settings; one token even pointed at another token’s contract. The scanner ran, found nothing to match, and raised nothing. A scan of the customer deposit addresses found nothing stranded, by luck more than design.
  • A scanner thread that dies quietly. Our block scanner restarts a chain’s runner whenever that currency’s settings change. The price oracle was touching the settings timestamp on every update, the restarted thread sometimes failed, and the failure was swallowed. We measured the cost afterwards: TRON detection down for 17.5 hours once, XRP for 3 hours 40 minutes twice, Ethereum 52 times for about 14 minutes each. A chain-stalled alert with a guarded automatic restart now catches it in about 15 minutes. The older “chain behind” alert would have taken about three and a half days.
  • A lag alert that cries wolf. The scanner deliberately stays the confirmation count behind the chain tip, so a plain “behind” alert flapped 807 times in 72 hours on TRON. Alert on lag minus the confirmation count instead.
  • An RPC provider that fails with HTTP 200. One provider answered “rate limited” inside a successful HTTP response, so the gateway saw a success and never failed over.
  • An app that never asked for an address. In the wallet app, a validator dropped every currency whose name or icon came back empty from the API. With no currency on the list, the app never requested a deposit address, and nothing logged why.

Monitor the scan cursor against the chain tip per chain, minus confirmations, and keep unrelated writes away from anything that restarts the scanner.

Credited, not swept, and labelled “Failed”

There are two thresholds, and they are easy to confuse:

  • the minimum deposit decides whether a deposit is recorded and credited;
  • the minimum collection decides whether it is worth sweeping to the hot wallet.

A deposit between the two is credited and deliberately left on the address. Our wallet apps showed those deposits as “Failed”, because they grouped those internal states under failure, while the customer’s balance was correct the whole time. On the operations side, the custody service raised the same deposit as a critical alert every five minutes for four days, because the wallet stack was running a version without the “dust” classification the exchange already had.

False criticals do more damage than missing ones: they teach people to mute the channel. “Not swept” and “not credited” need different states in the app, different alerts, and different words for support.

Credited, sweep failed: the gas tank nobody watched

To sweep a TRC-20 deposit, the platform first sends TRX from a fee wallet to the deposit address to pay for the transfer. The amount it sends is the transaction’s fee_limit times the number of legs in the sweep. On the exchange, the sweep split between the hot wallet and a trading venue, so it had two legs at 30 TRX each: 60 TRX per deposit, with each leg’s limit about 4.7 times what the transfer really burns. The fee wallet held less than that, the TRX transfer was rejected, and the deposit went to errored. The customer was credited; the coins stayed where they were.

What we changed:

  • fee_limit from 30 to 15 TRX on the exchange (the wallet went from 20 to 15 later, for the reasons in our TRON energy write-up).
  • A collection-gas alert that asks one question per chain: can the fee wallet pay for the next sweep? Its thresholds come from the platform’s own configuration, so they move when the configuration does: the need is fee_limit × legs, critical below 1.1 times that, warning below 2.5 times. If the configuration cannot be read, a separate “blind” alert fires instead of a silent pass.
  • Every errored deposit is raised as critical on its own.

One more trap: on the wallet stack the hot wallet and the fee wallet were the same address. Nothing in the platform stops that, and ordinary withdrawals then drain the gas that sweeps depend on.

“Collected”, but nothing moved on chain

This was the expensive lesson, and no customer noticed it, because crediting happens before sweeping. The treasury view was what went wrong.

The deposit daemon marked a sweep collected the moment the chain bridge returned a transaction hash. Nothing ever checked for a receipt; the status column on recorded transactions stayed pending for every single row. On Ethereum, an ERC-20 sweep goes out as two transactions back to back (gas top-up, then the token transfer). The bridge read the next nonce from the network for each send, with no lock per address, through a gateway that spreads requests across several public RPC providers. Two sends about a second apart could get the same nonce, and one of them disappeared. If the lost one was the gas top-up, the token transfer could never be signed at all.

What we measured once we looked:

  • About 29% of our Ethereum mainnet transactions over 60 days had no receipt.
  • Every pair of legs sent within a second of each other lost one leg. Every pair sent a minute or more apart landed.
  • On TRON, which has no nonce, every recorded transaction was on chain.
  • Withdrawals did have chain verification, and every withdrawal marked successful was confirmed paid.

We found it by accident in a treasury audit. It had gone unnoticed for about 19 days, and the longest any sum sat stranded on a deposit address was about 44 days.

The fix in the bridges for Ethereum, Avalanche and BNB Chain: a lock per sending address, a nonce counter that only advances after a successful broadcast, and a gas price floor of 0.5 gwei. The recovery of the stranded funds cost about $0.26 in gas in total.

The lessons:

  • A status column is not proof. “Sent” is only known from a receipt.
  • Reading the network’s pending nonce behind a load-balanced RPC pool does not serialize anything.
  • When you add a safety floor in one layer, every layer that pre-computes fees has to use the same number. We later found a case where the platform priced the gas top-up from the raw network price while the bridge signed at the floor: with network gas at 0.07–0.46 gwei that day, the top-up would have been 1.1 to 7 times too small. We caught that one before it stranded a deposit.

Dead ends that nothing retries

A state machine has states that nothing ever moves out of. We listed ours:

  • accepted is never retried. Processing is triggered at scan time only. We found one deposit that had sat in accepted for about a year. It was a ghost: a duplicate row for a transfer that existed once on chain and had already been swept under the other row. Forcing it through would have moved other customers’ coins under its name, so we muted it with a written reason instead.
  • errored is never retried automatically, which is the safe choice, since errored does not refund anything. It needs an alert, not a retry.
  • collected has no exit at all through the admin API.
  • A rescue block that logs and does not change state is an infinite loop. A dust deposit of a fraction of a cent of AVAX, sent from an address made to look like our hot wallet (address poisoning), was retried every 60 seconds for five weeks: 17,129 errors in one 12-day window. We stopped it by marking it skipped.
  • A Bitcoin wallet that cannot pay its own fee. Our Electrum integration ignored the “subtract fee from amount” flag, so when a deposit was the wallet’s only coin, its sweep stayed in processing forever. On the wallet stack, where the hot wallet and the deposit wallet are one wallet, we turned Bitcoin deposit collection off.

The alerts we run now are based on age per state: accepted for more than six hours, processing for more than 30 minutes as a warning and more than two hours as critical.

Confirmations: fast, but not faster than finality

On TRON we cut the confirmation count from 50 to 30 after measuring the distance between the chain head and the last solidified block: it held at 18 blocks. Deposits went from about 150 seconds to about 90. The platform has no reorg rollback, so the count can never go below finality; that is the floor, whatever competitors do.

We also found that the deposit screen’s “n of N confirmations” counter was broken: the list endpoint returned 0 instead of the real requirement for 49 currencies. Customers watching a counter that never moves open tickets.

What watches this now

Two tools, both read-only:

  • A reconciler that asks the chain about every recorded transaction and fails when one is not there. It knows the false alarms: chains that prune old history, and an Ethereum transaction without a receipt that is still in the mempool.
  • A sentinel that runs on a schedule and checks the claims the database makes: did “collected” funds actually leave the deposit address (balance at the block the database names), is every “sent” transaction on chain with the right amount (asked through a provider outside the normal RPC pool), were custody alerts delivered, how old are deposits in each state. When it cannot measure something, it raises that too: unmeasured is not the same as fine. It never sends funds and never writes to the database.

A reconciliation script that nobody schedules protects nobody. Ours existed for almost four weeks before the scheduled sentinel took over the job.

How we moved the stranded funds

Recovering coins from deposit addresses is a treasury movement with real keys, and we treat it like one:

  1. A read-only status of every address.
  2. A rehearsal: dry runs, plus negative tests that must fail (on TRON, 11 of them).
  3. One canary address, then an independent review that tries to prove the plan wrong.
  4. The remaining addresses one at a time. For each: a fresh balance reading, a dry run, a confirmation string computed separately in two places that must match, a human “yes”, exactly one transaction, and a check of the receipt (sender, recipient, amount, remaining balance) before the next address.

The gates around it: the destination can only be the hot wallet; the scanner must be caught up; the address must have had no incoming transfer for ten minutes; any gas top-up stays below 90% of the minimum deposit, so it can never be recorded as a customer deposit; and the database is compared before and after to prove nothing was written. An unknown outcome locks the step. Nobody resends until the chain says what happened.

Every transaction in both recovery runs came back with a receipt, and none ended unknown. The reviews caught two serious bugs before anything was sent: an integer overflow in a shell script on 18-decimal token amounts that would have silently disabled a whole phase, and the double-credit recommendation mentioned above.

One accounting note from the same work: some of the leftover TRX and ETH on deposit addresses had been booked as customer money in an earlier balance migration. Sweeping it was correct, but it was not revenue.

The short version

  • Make every deposit visible, including the ones below the minimum.
  • Separate “credited” from “swept” in your states, your alerts and your app.
  • Watch the gas tank that sweeps depend on, with thresholds derived from your own configuration.
  • Never mark anything sent without a receipt.
  • List every state that nothing retries, and alert on its age.
  • Reconcile against the chain on a schedule, and alarm when you cannot measure.

Deposit scanning, sweeping, the reconciliation and the alerts described here are part of Custody, and they run behind the Exchange and the Wallet. If deposits are your support desk’s biggest ticket category, talk to us.

დაჯავშნეთ 30-წუთიანი შეფასების ზარი

Crypto exchange, payments, wallet, custody and token launch software, already running in production. Run it on your own servers, license the source code, or let us host it under your brand.