All entries

    We Thought Notifications Were Duplicating. The First Diagnosis Was Wrong.

    A corrected notification audit reveals why subtype counts, atomic claims, delivery channels and product-wide limits are different debugging questions.

    Two servers competing to send one notification through an atomic claim gate to a phone above New York
    GPT-generated technical pixel-art illustration with a futuristic city backdrop. Conceptual engineering imagery, not a product screenshot or measured result.

    The most useful part of a postmortem is sometimes the correction to the first explanation. UltraFit360’s notification audit initially looked like a simple double-send story. On closer inspection, some repeated type labels represented different intended messages. Other counts did reveal over-firing, and a separate product problem remained: too many individually valid reminders could still feel like spam. That separation changed the investigation. This article explains the reasoning and documented partial fixes with fictional examples, without publishing user records, notification text or operational endpoints.

    First establish what one count means

    A group labelled “meal reminder” might contain lunch and dinner subtypes. Two rows under that label do not automatically mean the same message was delivered twice. Before diagnosing duplication, establish the unit: queued event, subtype, attempt, provider acceptance, device delivery or opened notification.

    The corrected audit matters because it went back to the underlying records and separated expected subtype activity from actual over-firing. I would preserve that correction in the public story rather than use the initial alarming count as a headline statistic. A misleading denominator can send a team toward the wrong code path for days.

    A read-then-write throttle can race

    Imagine two fictional workers checking whether today’s reminder was sent. Both read “not yet,” both queue the event and both update the count afterwards. Each worker followed the local rule. The combined system violated it because the permission to send was not claimed atomically.

    A process-local lock may help within one process while doing nothing across separate workers or instances. The protection must live at a boundary shared by the contenders. A database uniqueness or atomic-update pattern can turn “check then later record” into a single decision. The exact design should fit the event identity and failure semantics.

    Generic database pattern — illustrative, not a production migration
    INSERT INTO delivery_claims (event_key, calendar_day)
    VALUES (:event_key, :day)
    ON CONFLICT (event_key, calendar_day) DO NOTHING
    RETURNING event_key;
    
    -- Queue only if a claim was returned.
    -- External delivery still needs its own failure policy.
    EXPLAINED / SIDE BY SIDE

    Two workers can both pass a read-before-write check

    This ordering explains why a non-atomic throttle can allow duplicate work.

    Worker A

    1. 1 · Read eligibility

      No existing claim is visible.

    2. 3 · Decide to send

      Its earlier read still looked clear.

    3. 5 · Write the marker

      The marker arrives after the decision.

    Worker B

    1. 2 · Read eligibility

      A has not written its marker yet.

    2. 4 · Decide to send

      B also considers the event eligible.

    3. 6 · Write the marker

      Both decisions have already passed.

    THE KEY DISTINCTION

    Read the numbered steps across both workers. A later marker cannot retroactively make those decisions exclusive.

    Illustrative interleaving of two workers. The timing is explanatory, not a production trace.

    Claiming once is not the same as delivering once

    An atomic claim prevents some duplicate queuing, but external delivery has its own uncertainty. A worker can send a message and crash before marking it complete. Retrying may then repeat the external side effect. A provider response or receipt can establish a different stage from actual user-visible delivery.

    This is why I avoid a casual “exactly once” promise. Define the guarantees at each stage, use stable event identities where supported and reconcile ambiguous outcomes. The article’s lesson is not that one SQL pattern solves the entire delivery pipeline. It is that concurrency and external effects require separate reasoning.

    EXPLAINED / SYSTEM FLOW

    An atomic claim protects the decision boundary

    The database resolves competing claims; external delivery remains a separate failure boundary.

    Claim outcome

    1. Workers attempt the same unique claim

      The event identity and calendar scope must match the intended policy.

    2. Only a returned claim may enqueue

      A conflicting attempt does not create another queued send.

    3. Worker calls the delivery provider

      Track attempts and decide what recovery means after an ambiguous response.

    THE KEY DISTINCTION

    One successful claim is not proof of exactly-once delivery. A crash after provider acceptance still needs an explicit recovery policy.

    Companion to the generic SQL above. This model does not claim all duplicate-notification causes were fixed.

    Local and server reminders can overlap

    The audit also identified a server workout reminder that overlapped an on-device local reminder. Those messages came through different channels, so examining only the server queue could miss why a person saw two prompts about the same task.

    A useful inventory lists which component owns each reminder. The mobile device may schedule a personal local alert, while the backend sends a contextual message. Both can be valid features, but their overlap should be intentional. Removing an unnecessary duplicate channel is often simpler than adding another throttle around two systems that should not both send the same nudge.

    Per-type limits do not create a global product limit

    A meal reminder, step reminder and weekly check-in can each respect their own rule while adding up to a tiring day. That is a product coordination problem, not necessarily a concurrency defect. A person experiences the total interruption load, not the neat organisation of the job registry.

    The historical audit explicitly leaves some cross-type ceiling and locking work unfinished. The later engagement review also distinguishes active per-subtype protection from the absence of a fully enforced global cap. I would not claim that the team solved notification fatigue merely because one race was addressed. The remaining product policy deserves its own measurement and decision.

    Use the user’s calendar deliberately

    A “per day” rule needs a timezone definition. Counting a UTC calendar day while scheduling messages around local dinner can produce surprising boundaries. Daylight-saving changes and travel can add more edge cases. The right policy depends on the product, but it should not be an accidental consequence of the server’s timezone.

    For a test, use fictional accounts near a day boundary and run overlapping scheduler windows. Check eligibility, claim identity and the resulting queued events. Do not infer delivery from the presence of a row. A useful test names the stage it has established and leaves the rest to integration or device evidence.

    A reminder needs an outcome beyond being sent

    Delivery volume is easy to count. Helpfulness is harder. Did the person complete the intended action? Did they disable notifications? Was the reminder relevant to an actual missing action, or did onboarding-generated data trick the eligibility rule?

    I would review message targeting alongside the first meaningful task in the product. If returning users cannot find the logging action, another reminder may simply send them back into the same confusion. The frontend and lifecycle strategy should be considered together. A quieter, clearer flow may be more useful than a better-timed interruption.

    Preserve the partial nature of a fix

    The recorded implementation includes atomic-claim changes and removal of overlapping reminders. That is a legitimate engineering improvement. It does not prove the remaining global policy was deployed, that all users received fewer pushes or that retention improved as a result.

    A good follow-up records which invariants now hold, which remain open and how they will be observed. Use a synthetic event ledger in public explanations rather than raw customer records. Correcting the first diagnosis makes the story more credible: the team learned something important by checking the actual meaning of its data.

    Questions worth asking

    Can I solve duplicates with a boolean flag? Only if the transition is protected against concurrent readers and its failure behaviour is defined. A flag alone is not an atomic operation.

    Should every reminder share one daily cap? That is a product decision. Transactional alerts and optional nudges may need different policies, but exemptions should be explicit rather than accidental.

    What would I investigate first? Trace one reported experience through subtype, scheduling, queue and device channel. A person saying “too many notifications” may be describing a duplicate, a channel overlap or a valid but exhausting product policy.

    FURTHER READING

    Sources

    Technical references checked .

    END OF ENTRY / 19
    BEHIND THE JOURNAL

    Eeshan Singh

    I’m Eeshan Singh Pokharia, an AI Full Stack Developer and CoFounder of UltraFit360. I work across AI features, full-stack product development, and web and mobile experiences—from Figma and frontend interactions to the systems behind them.

    Notifications sit where backend scheduling meets the user’s patience. I want this story to preserve the audit’s correction and unfinished work, because a confident but oversimplified postmortem would teach the wrong fix.

    Meet the author