Article summary
Use a 100,000-phone TG job to design stratified inventory, a golden sample, sharding, wave expansion and capacity/quality acceptance before full scale.
One hundred thousand Telegram phones should not be treated as one file. They are a production job with a representative sample, shards, waves and stop conditions. Validate format, fields and exceptions before scaling. If reconciliation or error rate loses control at any stage, pause the next wave instead of accumulating results to meet a volume target.
Inventory the 100,000 first
Count country, source, acquisition month, customer stage and phone quality. Report raw rows, unique phones, missing country evidence, duplicates, suspect lengths and opt-outs. The unmodified “100,000” normally contains many rows that should never be submitted.
Draw a golden sample
| Sample part | Purpose | Composition |
|---|---|---|
| Normal stratum | Validate base schema | Main countries and sources |
| Difficult stratum | Observe exception handling | Legacy format, edge length, duplicate |
| Known stratum | Reasonableness check | Small internally known set |
| Veto stratum | Test process blocking | Opt-out and no-purpose records |
Task choice determines output width
TG Registration needs phone and registration result. Activity may include UserID, username, offline time, active days, names, VIP and frozen status. Gender and Age adds avatar URL, age and gender. At 100,000 scale, every extra column creates coverage, parsing and governance cost; do not default to the widest task.
Shard rules come before file size
Isolate platform task and country rules first, then split by source and risk. Each shard has fixed batch_id, unique input count, TXT hash and expected schema. Do not let one high-exception feed disrupt all others or mix tasks merely to create round numbers.
Expand in four waves
| Wave | Suggested scope | Release condition |
|---|---|---|
| 0 | Golden sample | Fields, states and joins correct |
| 1 | About 1% stratified sample | Exceptions explainable |
| 2 | About 10% production data | Throughput and review do not backlog |
| 3 | Remaining qualified shards | Budget and SLA still hold |
Control points in TXT generation
An AIPUSH TXT contains one phone per line with no header, name or business label. The generator emits crosswalk, row count, unique count, encoding and hash. A submitted file is immutable. Repair creates a new version and batch rather than overwriting the old one.
Accept end-to-end capacity
Measure P50/P95 time for cleaning, queue, processing, download, schema acceptance, reconciliation and human exceptions. Returning all phones while review backlogs for a week is not a capacity pass. service_rate ends only when usable observations are admitted.
Report quality by stratum
An overall reconciliation rate can hide a small country or dirty source. For every stratum, report explicit results, unknown, input_error, system_error, field nulls and mapping collisions. Stop expansion in a critical failed stratum rather than averaging it into a large successful one.
Recover only affected shards
Retry system failure with attempt_id, send format error back to normalization, quarantine a schema incident and do not endlessly rerun business unknown. An idempotency key prevents duplicate import of the same Excel, while every attempt preserves earlier responses.
Load-test human capacity
Measure minutes per hundred exceptions, daily collision closure and escalation ownership. If human backlog grows faster than it can be resolved, reduce the wave or narrow task fields immediately. Automated throughput is not a substitute for a staffing budget.
When the 100,000 job is complete
Unique input equals an explainable total of accepted, unknown, exception, error and excluded. Every shard has schema and hash; usable observations are timestamped in storage; temporary files have deletion dates. Completion is not a 100% progress bar—it is closed quantity and responsibility ledgers.
