Backend architecture

Shipping a PyQt5 desktop client: PyInstaller builds, an installer, an updater and a baked-in app key

A PyQt5 client for a Lightning Network node that charts the Bitcoin mempool live: PyInstaller release builds, an app key baked in at build time and why it is no secret, an installer, a SHA-256-checked updater, the sign-in/start/stop lifecycle and crash logging.

Qt and Python

I have been working on the client's desktop app for a Lightning Network node: a PyQt5 program its users install on their own Linux desktops. What they see is live Bitcoin mempool charts. What took most of the time was what a screenshot does not show: how the binary is built, how it reaches the user, how it updates itself, and what is left behind when it crashes.

The shape of the app

A two-step login window, then a main window with two tabs. The first holds the run controls, a network map (projected blocks and the last confirmed one), an activity log, and tables of recent transactions and RBF replacements; the second, mempool analytics, holds a two-hour incoming-transactions chart. Separate QThread workers poll on their own clocks (mempool summary every 25 seconds, tables every 15, chart statistics every 90, the backend heartbeat every 120), and none starts before the user runs the node. The data comes through a proxy on the Django backend rather than from mempool.space, which is filtered on some networks.

The chart worker turns the statistics into NumPy arrays of timestamps and vB/s, and a custom pg.GraphicsObject splits each segment where it crosses a colour threshold, so the colour changes exactly at the boundary. One pyqtgraph trap: before any data arrives the view spans decades of unix time, and a fixed 15-minute time axis produced about 1.9 million ticks there and froze the UI. The axis now coarsens its step past a tick limit, and both ranges are always set from the data.

Sign in, start, stop

Sign-in takes a username and an activation code, then a code sent by email. With the right code the server binds the device to a device_key and returns a long secret_code; both go into the OS keyring, and the next launch signs in silently through auto-login/. A run begins with start/, and the heartbeat is the same endpoint with status=update. The server counts online time only when heartbeats are at most four minutes apart, so the app sends one every two minutes, retries a failed one after 15 seconds, and gives up only after ten minutes without an answer. Every end of a run sends stop/, not just the STOP button.

All POSTs go through one function that encodes the body once, then signs and sends those exact bytes. A 403 carrying our API's JSON means “this build cannot talk to this server” and shows Update Required, never “wrong password”.

The auto-login that never worked

For a while every launch ended on the login screen. The cause was one line on the server:

product.set_secret_code()
product.device_key = device_key
product.is_app_login = True
# ...then, in response_to_login():
if must_update:
    product.save(update_fields=["app"])

The response was built from the in-memory object, so it was a clean 200 with a fresh secret, while only the app column reached the database. The narrowing itself was deliberate (the crash log is appended by a queryset UPDATE, and a bare save() would write a stale copy over it); the field list just never grew. Now each caller declares what it changed, and the tests assert against the reloaded row, not the response. Three client faults had hidden it, including a refused secret retried forever because detection matched on error text.

The release build

Releases come only from build.sh, run inside a pinned Docker image based on Ubuntu 22.04. It builds three --onefile binaries (the app, the updater and the installer), and only the app carries the key:

cleanup() {
    $PY bake_secrets.py --clean || rm -f _embedded_secrets.py
}
trap cleanup EXIT

$PY bake_secrets.py
$PYINSTALLER --onefile --noconfirm \
    --name <app> \
    --collect-all PyQt5 \
    --hidden-import _embedded_secrets \
    --hidden-import socks \
    --hidden-import urllib3.contrib.socks \
    main.py

_embedded_secrets is imported inside a try/except, so PyInstaller cannot see it; drop the --hidden-import and the build quietly ships without a key. So the script ends by checking PYZ-00.toc for the module and fails if it is missing. The socks imports serve users whose VPN is the system's SOCKS proxy.

The app key, and why it is no secret

A plaintext .env next to the executable is readable, copyable and easy to repoint, so it never leaves the build host. bake_secrets.py reads the values from the environment first and .env second, writes them into a generated module XOR-ed against a random per-build mask, and the trap deletes that module on exit, whether the build succeeded or not:

def obfuscate(value, mask):
    raw = value.encode("utf-8")
    blob = bytes(b ^ mask[i % len(mask)] for i, b in enumerate(raw))
    return base64.b64encode(blob).decode("ascii")

That is obfuscation, not encryption: the mask ships beside the blob. It beats strings, not someone who unpacks the binary. Every install shares the key, so one extracted copy works forever, and rotating it would lock out every installed build. By design, then, the key is a build identifier, not a trust anchor; real privilege sits with the device_key, and a per-device secret is specified but not built yet.

The HMAC scheme limits what an intercepted request is worth: the signature covers path:timestamp:sha256(body) and the timestamp is valid for 300 seconds, so a captured signature cannot carry a different payload. The cost is that a machine whose clock is five minutes off cannot reach the backend at all.

The installer

The real installing is done by a bash script already hardened on users' machines (APT locks, mirror switching with rollback, a filtered server), so the installer is a PyQt5 window around it rather than a rewrite. The window runs the script with --ui=pipe and reads stdout: plain lines are the log, and @@ lines (STEP, PHASE, DONE and others) drive the progress page; the sudo password goes through askpass and a FIFO, never to disk.

Before installing, the script checks the zip header, tests the archive and compares its SHA-256 with the published digest; afterwards it writes a sha256 manifest, so a later run can tell a complete installation from one with a file missing or replaced, and Repair puts it back. The installer carries no key and does not import config, so it opens at once even on a filtered network.

The updater

A new version shows up in the answers to login/, auto-login/ and start/ as new_version, force_update and the latest version string. An ordinary update can be postponed for a few days; a forced one offers only Download or Exit. Download starts the updater binary in its own session and quits the app. The updater carries no key either: latest/ and latest/sha256/ are open on purpose, because a build that fails the key check must still be able to fetch one that passes.

expected = self.fetch_expected_checksum() or (
    response.headers.get("X-Checksum-Sha256", "").strip().lower()
)
if expected:
    actual = self.file_sha256(self.save_path)
    if actual != expected:
        self.cleanup_partial_download()
        self.finished.emit(False, "... (checksum mismatch) ...")
        return

A mismatch deletes the file and fails the update. If no digest is available at all, the updater logs it and carries on; the install script has a --strict flag for that case and the updater does not, a gap that is still open. Then it closes running copies of the app with psutil, moves the current binary aside with a date suffix, renames itself to .old so a new updater can land, extracts behind a commonpath guard against path traversal, sets the execute bit and launches the new version. Any failure puts the backup and the updater's name back.

Crash logging

A crashing node is not reliably reaching the server, so nothing is reported live. app_log.py is the first thing main.py installs, before the imports that can fail, and it banks sys.excepthook, threading.excepthook, Qt's critical and fatal messages and faulthandler output in a file next to the executable. The faulthandler file handle stays open for the whole run, since a signal handler cannot open a file, and the next launch folds the dump into the buffer.

The buffer, capped at 20,000 characters, rides along as a form field on the next lifecycle call and is cleared only by a 2xx that is JSON; a captive portal answers 200 too. Nothing in the module may raise: a logger that can crash the app it explains is worse than none.

What I took away

  • A key inside a client is a build identifier; bind authority to something issued per device.
  • A build should verify itself, or one day it will ship without its key.
  • Test the stored row, not the response.
  • The updater and the installer must work without the main app, on the worst network.