Skip to content

Multi-tenant ERP

Mohaseb

Cloud accounting and point-of-sale for many companies, working offline and serving more than 500 businesses, with Arabic and English interfaces.

Year
2024–2026
Role
End-to-end design and development
Category
Multi-tenant ERP
Mohaseb: Mohaseb: Multi-tenant ERP

The challenge

Many small businesses needed one platform, each with its own books, users and a point of sale that keeps working when the internet drops. The available options were either local systems with no central books, or rigid cloud suites that handle Arabic poorly.

The approach

A Laravel accounting core and tenant API, an offline-first Flutter point-of-sale app with local storage, and a WebSocket layer that syncs cashier and back office in real time, with idempotency keys on every write.

The outcome

The system runs today for more than 500 businesses, with Arabic and English interfaces and a point of sale that does not lose transactions when the network drops.

Under the hood

Field
Accounting Tech · Multi-tenant SaaS · POS · E-commerce
Type
Multi-Tenant ERP + Storefront Platform
Team
end-to-end (architecture, 4 web apps, 2 mobile apps, backend, DevOps)

01

The real problem

Small businesses across the GCC were losing orders during internet outages, then losing hours every week reconciling cash drawer vs. server records the next morning. Existing cloud ERPs (Zoho Books, QuickBooks, Wafeq) either refused to operate offline or kept a "dumb cache" that silently dropped transactions on reconnect. Local desktop POS systems (Foodics on iPad, ETAP, local Excel sheets) worked offline but had no central books — every branch became its own island, and a single chain across three cities could not produce one trustworthy P&L. ZATCA Phase 2 e-invoicing landed in 2024 and broke half of the local POS vendors overnight. The brief: build the first system in this market that is simultaneously offline-first, multi-tenant, accounting-grade, and ZATCA-compliant out of the box — no compromise on any of the four. And ship it as a single ecosystem (landing → onboarding → admin → super-admin → cashier POS → manager mobile → tenant storefront → public REST API), not five disconnected products glued together.

02

Who did what

End-to-end build of a multi-tenant ERP serving 500+ live businesses. The 7 named hats below describe what it takes to ship and operate it.

Product architect
Full domain model: 78 tables across 24 modules (accounting, inventory, HR, payroll, POS, manufacturing, projects, CRM). Tenant isolation strategy, sync hierarchy, ZATCA Phase 2 integration design.
Backend engineer
Laravel 11 monolith, 200+ controllers, 600+ Eloquent models. Double-entry accounting engine, weighted-average cost stock valuation, multi-currency with auto FX adjustment entries, payroll engine, manufacturing BOM resolver.
Frontend × 4 web apps
Tenant app (Livewire + Alpine), super-admin (Livewire), landing site (Next.js 14), storefront builder (Next.js 14 + ISR). All bilingual ar/en with native RTL — never machine-translated.
Mobile × 2 Flutter apps
Cashier POS (offline-first via Drift) and Manager mobile (approvals + branch comparison). Riverpod state, sync engine with idempotency keys, 48-table local schema mirror.
Sync engine designer
Three-tier hierarchy: WebSocket (Reverb) primary, Unified Pull secondary, polling fallback floor. Five-layer zero-duplicate guarantee. UUID-to-server-ID rewriter at push time. Delta cursor with FK-skip safety.
Database designer + DevOps
500+ MySQL schemas via Stancl tenancy, per-tenant migrations, Redis 7 (4 DBs: cache/session/queue/broadcast), Horizon + Supervisor, nginx 1.26 routing, daily mysqldump backups, restore-tested monthly.
UI/UX designer + brand
Every screen on all 6 apps. Tajawal/Readex Pro Arabic system, light + dark, dense data tables with editorial typography. Brand identity, logo, marketing copy, Arabic UX patterns from scratch.
QA + on-call ops
20-scenario hostile test matrix (cross-tenant attack, 10k offline push, network flap, clock skew, Redis down, …). Canary tenant rollouts. On-call for every incident on this platform for 18 months running.

03

Architecture

01 — Edge

Edge tier

  • Cloudflare

    CDN · WAF · DDoS

    Country-level routing, AI-bot block, signed Cache-Control on /index.html

  • Nginx 1.26

    Reverse proxy · TLS

    Per-tenant subdomain → app upstream, /app* reserved for Reverb WS

02 — Clients

Client tier (4 web · 2 mobile · 1 public API)

  • Landing site

    Next.js 14 · App Router

    Marketing, signup, ZATCA briefing pages

  • Tenant app

    Laravel + Livewire + Alpine

    Daily accounting, POS-web, reports, settings

  • Super-admin

    Laravel + Livewire

    Tenant CRUD, feature flags, observability dashboards

  • Storefront builder

    Next.js 14 · ISR

    Per-tenant public storefronts, edge-rendered on Vercel

  • Cashier POS

    Flutter · Drift · Riverpod

    Offline-first; UUID FKs rewritten at push time

  • Manager mobile

    Flutter · Drift · Riverpod

    Approvals, branch comparison, push notifications

  • Public REST API

    Laravel API + Sanctum

    Documented + tested; tenant scopes via Stancl middleware

03 — Application

Application tier (Laravel 11 monolith — tenant-aware)

  • Tenant routing

    Stancl/Tenancy v3

    DB switch per request, soft-deletes + try-catch migrations

  • Accounting engine

    Double-entry · Odoo-grade

    Debit=credit enforced in tx; tax inclusive/exclusive; WAC stock

  • ZATCA Phase 2

    XML + cryptostamp + QR

    Real-time clearance for B2B, reporting for B2C — fully compliant

  • Permissions

    Spatie/permission · per-tenant

    Role-scoped account reads; admin override audited

  • Sync gateway

    UnifiedSyncController

    Single endpoint replaces 48 per-entity calls; LSN-aware delta pull

04 — Async

Async + real-time tier

  • Redis 7

    Cache · Pub/Sub · Sessions

    Separate DBs for sessions / cache / queue / broadcasts

  • Horizon + Supervisor

    Redis-backed queue workers

    Auto-scale; per-tenant queue tags for fair scheduling

  • Reverb (Laravel WS)

    WebSocket broadcaster

    Real-time sync; per-tenant channels; feature-flagged rollout

  • SyncBroadcastObserver

    Eloquent observer

    Auto-emits create/update/delete events for 13+ entity types

05 — Data

Data tier

  • MySQL 8

    Per-tenant schema

    500+ databases, sql_<uuid> naming, online migrations via Stancl

  • Drift (SQLite)

    Flutter local DB

    48 tables mirror tenant schema; partial-unique indexes on server_id

  • Object storage

    Local disk + nightly backup

    Tenant uploads at /storage/tenants/<id>; signed URLs only

  • Backups

    mysqldump + retention 7d

    Daily per-tenant + weekly full; restore-tested monthly

06 — Observability

Observability + safety

  • Sync logs

    broadcast_events_log + sync_logs

    Every push/pull traced; idempotency_key + actor + tenant

  • Divergence scanner

    Nightly job

    Hashes F↔S per entity; >0 = page; root-causes investigated within 24h

  • Feature flags

    tenants.* columns

    Per-tenant WebSocket toggle; instant rollback without redeploy

  • Audit log

    AuditService · 7y retention

    Every mutation tagged with actor + IP + tenant + reason

04

Decisions that mattered

The interesting part is not the API surface. It is what happens when a cashier rings up an invoice while their connection is dropping every fifteen seconds.

  1. 01

    Local UUIDs for foreign keys at write time, rewritten to server integer IDs only at push time. Removes the entire class of "offline contact never synced" bugs.

  2. 02

    Touch the delta cursor only after at least one row was upserted. Skipping this rule loses data on FK-skipped batches.

  3. 03

    Idempotency key on every push, mirrored on the server, retried under the same key until acknowledged.

  4. 04

    Reverb for real-time, polling fallback under feature flag, never both at once. Old Flutter versions keep working because polling is the floor, not the ceiling.

05

Challenges

  1. 01

    Offline-first synchronization with three-way conflict resolution across cashier POS, mobile manager, and back-office web apps running on flaky 3G/4G networks. Each client maintains its own local source of truth and reconciles on reconnect without operator intervention.

  2. 02

    Multi-branch consistency: a stock transfer initiated in Branch A must never count twice when Branch B comes back online seven hours later. Implemented with version-vector optimistic locking and a server-authoritative reconciliation pass.

  3. 03

    Real-time kitchen, inventory, and order updates over WebSockets that degrade gracefully to polling for older Flutter app builds still in production phones. Three-layer sync hierarchy: WebSocket primary → Unified Pull secondary → legacy polling floor.

  4. 04

    Strict multi-tenant isolation with per-tenant MySQL database (not shared schema with tenant_id column). One tenant cannot read another tenant's rows under any code path — even under JWT-claim spoofing, worker queue race, or developer error in custom queries.

  5. 05

    ZATCA Phase 2 e-invoicing compliance: cryptographic invoice hashing with previous-invoice chain, QR code with TLV-encoded VAT data, B2B clearance API integration, B2C reporting API integration, all bilingual (Arabic-first).

  6. 06

    High concurrency: up to 1,500 simultaneous cashier sessions during lunch rush across all tenants.

  7. 07

    Server-side document number generation that produces stable invoice numbers even when 50 offline devices push their backlog at the same moment. Per-series, per-device reservation pool with TTL-based release.

  8. 08

    Backward compatibility forever — old Flutter app versions in the field must keep working as the server evolves, because remote shops cannot force-update phones. Every API endpoint is versioned and old versions live for ≥18 months past deprecation.

  9. 09

    Tenant provisioning under one minute: from "sign up" click to "first invoice issued" must include creating MySQL database, running 80+ migrations, seeding default chart of accounts (English + Arabic), VAT configuration, sample customers, user roles, default templates, and routing the storefront subdomain.

  10. 10

    Subscription billing engine with mid-cycle proration: upgrade from Starter to Pro on day 17 must credit the unused 13 days of Starter and charge a prorated 13 days of Pro — and produce a tax invoice that survives an auditor.

  11. 11

    Drift (Flutter local SQLite) schema migrations on a phone that may be 30 versions behind. Migrations must be incremental, idempotent, recover from partial application, and never silently drop user data.

  12. 12

    POS hardware integration on Android: USB receipt printers (ESC/POS), USB cash drawer kick, USB barcode scanners (HID + serial modes), Bluetooth weighing scales, iZettle/Geidea card terminals.

  13. 13

    Multi-tenant cache invalidation: when a tenant edits a product, the storefront Next.js page (per-tenant ISR) must invalidate within seconds, but a sibling tenant's cache must stay untouched.

  14. 14

    Live tax engine swap when KSA, UAE, and Bahrain rules change — without redeploying the API or breaking historical invoices.

06

Hard problems

  • Offline sync engine with FK rewriting

    A cashier creates a new customer mid-invoice while offline. The customer row gets a local UUID; the invoice references that UUID; payments + journal entries reference the invoice UUID. On reconnect, a topologically-sorted push engine walks the dependency graph: customer first, captures the server integer ID, rewrites every dependent reference (whitelist of 14 FK columns including contact_id, supplier_id, account_id, product_id, order_id, voucher_id) before pushing them. Each row carries an idempotency key mirrored on the server; a half-acknowledged batch never duplicates on retry. Eliminated the entire "offline contact never synced" bug class without a single hand-rolled deduplication query.

  • Inventory race conditions across branches

    Two cashiers in two branches sell the last unit simultaneously. Solved with optimistic locking: the inventory row carries a version column; the push payload carries the version it was based on; the server rejects the write if the version moved. The cashier app surfaces a "stock changed — confirm" dialog instead of silently overselling. Combined with reserved-quantity logic during the open transaction window and a nightly stock-reconciliation cron, oversell incidents dropped to near zero across 500+ tenants.

  • Stable invoice numbers across offline pushes (ZATCA-grade)

    ZATCA requires monotonic invoice numbers without gaps AND each invoice cryptographically chained to the previous one. Offline devices push out of order. Moved number allocation to a server-side NumberPool service that reserves a number per device per series with TTL, commits on first successful push, releases on TTL expiry if the device failed mid-batch. The chained hash is computed server-side after number commit, so the chain stays unbroken even when device order is wrong. Zero gaps, zero duplicates, zero "invoice number changed after sync" complaints in 14 months of ZATCA Phase 2 compliance.

  • Five-layer Zero-Duplicate Guarantee

    Every transactional table is protected by five independent layers, so a duplicate row requires all five to fail simultaneously. L1: UNIQUE partial index on server_id (DB-level physical guarantee). L2: push integrity — orphan rows retry under the same idempotency key, never silently succeed. L3: pull idempotency — re-pulls UPDATE existing rows, never INSERT. L4: business-key stability — offline invoice number = online invoice number forever. L5: nightly post-write DivergenceScanner that flags any row whose hash differs between local and server. Mathematically negligible duplicate probability, proven across 2.3M synced rows.

  • Unified Pull replacing 48 per-entity sync endpoints

    The first version made 48 separate API calls on every cold-start sync (one per entity: products, categories, units, taxes, customers, suppliers, accounts, orders, vouchers, payroll, …), each with auth + rate-limit + TLS handshake overhead. Most returned empty. Replaced them with one POST /sync/pull that takes a per-entity cursor map and returns one streamed response per entity with delta + tombstones. Cold-start went from 4.8s avg to 0.9s on 4G; the Cloudflare WAF stopped flagging the burst as suspicious; mobile data consumption dropped ~70% per cashier per day.

  • Per-tenant DB provisioning

    Sign-up flow: create MySQL database, set per-tenant DB user with scoped privileges, run 80+ migrations idempotently (each wrapped in try-catch so re-runs on partial failure are safe), seed default chart of accounts (12 root nodes × 2 locales × KSA/UAE/Bahrain variants), seed VAT codes, seed user roles + admin user, seed default invoice / receipt / quote templates, route the tenant subdomain in the storefront, fire welcome email. All as a Horizon job chain with per-step rollback. Failure modes: every step is resumable from any point.

  • Subscription billing with mid-cycle proration

    Upgrade from Starter to Pro mid-cycle must credit the unused days of Starter and charge the remaining days of Pro, producing a single tax invoice that survives a ZATCA audit. Built a Decimal-based proration engine on top of Stripe + local gateways, handling: mid-cycle plan changes (up or down), seat additions, add-on activations (extra branches, extra warehouses), payment retries with backoff, dunning emails, and graceful downgrade-on-failure. Tax is calculated on the prorated amount per jurisdiction, then split across the original invoice and a credit note when proration creates a refund.

  • Drift schema migrations on phones 30 versions behind

    A cashier upgrades the app after 8 months — local DB is at schema version 12, app expects version 42. Wrote a migration runner that walks every intermediate version in order, each migration idempotent and crash-safe (uses ALTER + COALESCE patterns, never DROP), records progress in a meta table so a power-cut mid-migration resumes from the last committed step. Tested across 30+ artificial leaps to ensure no path through the version graph silently drops data.

  • POS hardware integration on Android tablets

    USB receipt printers (ESC/POS) — 3 vendors, each with quirks; the print queue is its own Riverpod-managed isolate that survives Activity restarts. USB cash drawer kick — fired via printer escape sequence. USB barcode scanners — auto-detect HID vs serial mode. Bluetooth weighing scales — paired once, polled by an isolate that reconnects on Bluetooth flap. Card terminals (Geidea, Foodics-bundled, iZettle) — integrated via vendor SDKs, with a fallback "manual amount" mode when the terminal is offline. All hardware events go through a single OrderBus event sourcing layer so a hardware failure never blocks an open invoice.

  • Per-tenant cache invalidation across the stack

    When tenant X edits product P: (a) tenant X's Laravel cache invalidates the products list key, (b) tenant X's Next.js storefront triggers an ISR revalidation for /products and /products/P, (c) tenant X's POS app receives a Reverb broadcast and refreshes its local Drift row. Tenant Y's caches stay untouched. Implemented with cache tags namespaced by tenant_id; the storefront uses Next.js on-demand revalidation API with a per-tenant secret to prevent cross-tenant cache poisoning.

  • Feature-flagged WebSocket rollout per tenant

    Could not ship Reverb to all tenants at once — too risky. A per-tenant boolean `websocket_enabled` gates whether the broadcast observers dispatch events. Old Flutter clients keep polling (fallback floor). New ones receive real-time updates the moment we flip the flag. The flag is super-admin-toggleable from a single dashboard cell — partial rollback takes one second, not a deploy. The 3-layer hierarchy (WebSocket → Unified Pull → legacy polling) means a Reverb outage degrades to a 60-second sync, not a dead app.

  • Cross-system tenant SSO

    A user logs in to admin.X.mohaseb.co and then clicks "open my storefront" — they land in the Next.js tenant store as an authenticated admin (so they can preview unpublished products). Built a signed-handoff token: Laravel issues a short-lived JWT scoped to "storefront preview", redirects to Next.js, Next.js verifies the signature, exchanges for a session cookie scoped to that subdomain. Same pattern handles cashier → manager app on the same tablet, super-admin → tenant impersonation (with audit log entry on every impersonation).

07

Why these technologies

  • Laravel 11 (backend)

    • Mature multi-tenant ecosystem (Stancl/tenancy for per-tenant DB, Spatie/permission for RBAC, Spatie/medialibrary for tenant uploads) — all production-grade with active vendor support.
    • First-class WebSocket support via Reverb in Laravel 11 — no extra Node service to babysit; same auth context end-to-end.
    • Eloquent global scopes make the audit-log + soft-delete + per-tenant scoping invariant trivial to enforce at the model layer, not scattered across queries.
    • Queue + Horizon + Mail + Notifications are first-party, not third-party glue — every cross-cutting concern uses the same idioms.
    • Job batching + chains express the "provision a tenant" flow naturally: create DB → migrate → seed → notify, with rollback on any failure.
  • Livewire + Alpine.js (admin UI)

    • Server-rendered with reactive updates — no separate SPA build pipeline, no API duplication for forms, no client-side state management overhead.
    • Single developer can ship full-stack features (model + migration + Livewire component + Blade view) in one PR without context-switching.
    • Form validation, file uploads, and table sorting come for free with Livewire — what would be 300 lines of React + axios is 40 lines of Blade.
    • Renders fast on cheap shop computers (no JS framework hydration cost).
  • Next.js 14 (tenant storefronts)

    • App Router middleware reads the tenant subdomain once and injects context into every page — no per-route boilerplate.
    • Image optimization, automatic code splitting, edge runtime support — features I would otherwise re-implement.
    • SEO matters for storefronts: SSR + structured data + sitemap generation are baked in.
  • Flutter + Drift + Riverpod

    • Single Dart codebase ships Android (primary), iOS, macOS, and Windows with the same business logic. POS hardware drivers live in platform channels but the rest is shared.
    • Drift gives compile-time-typed SQLite queries — sync code stays safe across schema evolutions; the Dart analyzer catches column drift at build time.
    • Riverpod's family providers + auto-dispose handle the "10 cashier sessions open at the same time, each with its own state" pattern cleanly.
    • Hot reload on a real cash register over USB makes iterating on receipt layouts fast.
    • Tested down to "the printer driver returns junk bytes at 14% battery" — Flutter's isolate model lets us crash-and-restart the print queue without dropping the open invoice.
  • MySQL 8 (per-tenant DB)

    • JSON columns simplify multi-locale fields without giving up relational integrity — search the JSON, foreign-key the row.
    • CTEs and window functions handle accounting period-close queries (trial balance, P&L, balance sheet) elegantly.
    • Per-tenant database isolation means a single noisy tenant cannot impact others — and a tenant export is just a mysqldump of one DB, not a tenant_id filter on 80 tables.
    • Mature backup / restore tooling, well-understood replication, every ops engineer in MENA can read it.
  • Stancl/Tenancy

    • Switching the current tenant is a middleware concern — every Eloquent query inside a request automatically talks to the right database.
    • Built-in tenant lifecycle hooks (create / delete / soft-delete with grace period) save weeks of bespoke work.
    • Per-tenant Redis namespace, queue tag, storage prefix, cache key prefix — all enforced by the package, no developer discipline needed.
  • Redis + Horizon

    • Reliable queue backpressure during the noon rush when 1,500 cashiers push at the same minute.
    • Horizon UI surfaces stuck jobs and per-queue metrics without writing a custom dashboard.
    • Pub/sub is the natural transport for Reverb broadcasts — same Redis instance, no extra moving part.
    • Per-tenant queue namespacing prevents one noisy tenant's billing webhook retries from starving another tenant's POS sync.
  • Reverb (WebSocket)

    • In-process with Laravel — same auth context, no separate token exchange, no separate Node service to monitor.
    • Pusher-protocol compatible — older Flutter clients using pusher-channels-dart kept working with zero code change after migration.
    • Per-tenant channel namespaces make access control declarative: a subscribe attempt to tenant.X.orders from a session authenticated to tenant.Y returns 403 at the channel layer.
    • Horizontally scalable behind a Redis adapter when one tenant needs multiple instances.
  • Nginx + Supervisor (instead of Docker)

    • Battle-tested combo on Debian — no Docker overhead on a small VPS where every CPU cycle is paid for in dollars per month.
    • Supervisor restart policies handle Reverb / Horizon crashes within seconds, with throttling so a crash loop does not consume the box.
    • Single-file Nginx config per tenant subdomain — easy to audit, easy to grep, easy to roll back.
    • No container orchestration complexity for a system where the deploy story is "git pull + composer install + supervisor reload".

08

Security and isolation

  • JWT auth with 15-minute access + 30-day refresh, refresh rotation + reuse detection — a stolen token triggers a whole-session-tree wipe and forces re-login on every device.
  • RBAC with 50+ permissions composed into roles; ADMIN bypass kept explicit and audited; super-admin impersonation logs an audit entry on entry AND every action taken while impersonating.
  • Argon2id password hashing (PHP password_hash with explicit cost) — no MD5/SHA legacy, no PBKDF2 shortcut. Password policies enforced server-side, not client-side.
  • Per-tenant strict isolation enforced at the database-connection layer (each request swaps DB credentials to the tenant's scoped user) plus an Eloquent global-scope safety net. Cross-tenant attack simulations run on every deploy.
  • Per-tenant DB user has scoped GRANTs (SELECT/INSERT/UPDATE/DELETE only on its own DB — no information_schema, no other DBs, no FILE privilege).
  • Audit log on every mutation (who, what, before/after JSON diff, where, when, IP, UA, request_id) — queryable by tenant admin, immutable, retained 7 years for compliance.
  • Rate limiting per-route via Laravel Throttler — public endpoints 60/min/IP, contact forms 5/min/IP, login 5/min/IP+username, password reset 3/hour/email.
  • WebSocket auth signs every channel subscription with the tenant ID + user ID; cross-tenant channel attempts return 403, logged, alerted at 10/min threshold.
  • CSRF tokens on every state-changing form; double-submit cookie pattern + same-site=lax on session cookies.
  • Daily per-tenant mysqldump + gzip with 30-day retention, weekly full landlord dump, monthly off-site rotation, quarterly restore drills (timed, documented).
  • Encrypted sensitive columns (SMTP passwords, OAuth secrets, Stripe API keys, ZATCA certificates) at the application layer using Laravel's built-in encrypter with per-tenant keys.
  • ZATCA cryptographic invoice signing using per-tenant certificates issued via the ZATCA portal, with chain-of-trust validation on every invoice.
  • File upload validation: MIME sniffing (not extension trust), magic-byte check, max-size per type, virus scan hook, S3-prefix isolation per tenant.
  • CSP headers + Strict-Transport-Security + X-Content-Type-Options + X-Frame-Options + Referrer-Policy + Permissions-Policy set on every response via Nginx.
  • Webhook signatures verified on every inbound (Stripe HMAC, ZATCA RSA) — no idempotent webhook ever processed twice (idempotency key + 30-day replay protection).

09

Scale

Designed for horizontal scale on the stateless API tier (Laravel workers behind Nginx, no session affinity), with a single primary MySQL writer per tenant. The single-writer bottleneck is intentional — accounting integrity beats raw throughput; an accountant who sees double-charges loses trust permanently. Scale-out comes from sharding tenants across multiple writer instances (already implemented via Stancl multi-DB but not yet load-required), not from multi-master fan-out which would compromise the ZATCA cryptographic invoice chain. Storefront tier scales independently on Vercel-style edge ISR with per-tenant cache tags. Reverb scales horizontally behind Redis pub/sub when a single tenant's real-time load needs more than one Reverb instance.

  • Multiple countries — KSA, UAE, Bahrain (separate VAT engines)
  • Multi-brand / multi-branch under one tenant
  • Horizontal API scale (stateless workers behind Nginx)
  • Per-tenant MySQL shard for noisy neighbours
  • Per-tenant Redis namespace for cache + queue isolation
  • Per-tenant Reverb channel namespace (cross-tenant isolation enforced)
  • Storefront edge caching with per-tenant invalidation tags
  • Per-tenant rate limits prevent one shop's API misuse from affecting others
  • Job queue per-tenant partitioning prevents billing webhook storms from starving POS sync
  • 24-hour offline cashier session before forced re-sync (configurable per tenant)

10

What I learned

  1. 01

    Offline sync complexity is consistently underestimated by a factor of 3. Plan triple the time you think it needs, and write the divergence scanner before you write the sync engine — not after. The scanner is your only honest test for correctness.

  2. 02

    Feature flags scoped per-tenant are the difference between confident rollouts and weekend incidents. Spend the day setting up the flag table BEFORE you ship the feature behind it. Every non-trivial change ships gated, then ramped, then made default.

  3. 03

    Idempotency keys at every layer (HTTP request, push payload, queued job, webhook handler) are not paranoia — they are the only reason the system is not a duplicate-row factory. The cost is one extra column per table; the benefit is "we will never bill twice".

  4. 04

    Multi-tenant isolation must be enforced at the LOWEST layer that can enforce it. Application-layer scoping plus per-tenant DB credentials plus per-tenant DB GRANTs is belt + suspenders + airbag — not redundancy. A junior dev forgetting `withGlobalScope` cannot leak data across tenants because the DB credential physically cannot see the other DB.

  5. 05

    Backward compatibility is non-negotiable when you cannot force-update the client. Every API endpoint lives 18 months past deprecation. Old Flutter app versions in remote shops still work because the server treats every version as a first-class citizen, not a deprecation candidate.

  6. 06

    WebSockets without a polling fallback is a fragility you will regret. The 3-layer hierarchy (WebSocket → unified pull → legacy polling) means a Reverb outage degrades to a 60-second sync, not a dead app. The polling fallback is the floor, not the ceiling.

  7. 07

    The Five-Layer Zero-Duplicate Guarantee took two weeks to design. Math is cheaper than support tickets.

  8. 08

    Server-side number generation (NumberPool) is the only correct answer when offline devices must produce gap-free monotonic numbers. Client-side hand-rolled "INV-{timestamp}" feels clever until the first ZATCA audit fails because invoice 142 went out before invoice 141.

  9. 09

    Per-tenant DB beats shared schema for any system that has to give a tenant their data back. Export is a mysqldump, not a 200-line custom export script with edge cases. Delete is DROP DATABASE, not a 20-table CASCADE that you pray finishes before timeout.

  10. 10

    Livewire for admin + Next.js for storefronts + Flutter for mobile is not "three stacks" — it is three stacks for three audiences. Forcing a single SPA across all of them would have added 6 months and saved nothing.

  11. 11

    Hardware (printers, drawers, scanners) is a common failure point. Wrap every hardware call in a circuit breaker; surface "printer offline — invoice saved, will reprint on reconnect" to the cashier instead of crashing the open order.

  12. 12

    Keeping complexity honest on the critical path is a core rule. If a feature requires three days of mental load to hold in your head, it is too complex — refactor before shipping. The system survived because nothing inside it is cleverer than necessary.

Stack

  • Laravel
  • Flutter
  • MySQL
  • Redis
  • WebSockets
  • Offline-first sync

Questions about Mohaseb

Who built Mohaseb?

Mohaseb was delivered by Ali Haitham Yosef, founder of a software company in Damascus that carries his name. His role on the project: End-to-end design and development.

What is Mohaseb?

Cloud accounting and point-of-sale for many companies, working offline and serving more than 500 businesses, with Arabic and English interfaces.

What technology does Mohaseb use?

Laravel, Flutter, MySQL, Redis, WebSockets, Offline-first sync.

When was Mohaseb built?

It was built during 2024–2026.

Can you build a system like Mohaseb for my business?

Yes. The company builds ERP systems, point-of-sale systems, mobile apps and websites for clients in Syria, Lebanon, Jordan, Iraq, Turkey and the Gulf states, working remotely. Contact: info@aliyosef.online.