Eric Guo's blog.cloud-mes.com

Hoping writing JS, Ruby & Rails and Go article, but fallback to DevOps note

Upgrade PostgreSQL 16.x to 17.x on Rocky Linux 9.2

Permalink

This guide upgrades a dedicated Rocky Linux 9.2 server from PostgreSQL 16.x to 17.x in place with pg_upgrade. PostgreSQL 17 is installed alongside PostgreSQL 16, and the old data directory is retained for rollback.

The commands use --copy, not --link. Copy mode requires enough free disk space for another copy of the cluster, but PostgreSQL 16 remains usable until PostgreSQL 17 receives new production writes.

The paths below match installations made with the official PostgreSQL Yum repository:

Set the PostgreSQL upgrade paths
export OLD_BIN=/usr/pgsql-16/bin
export NEW_BIN=/usr/pgsql-17/bin
export OLD_DATA=/var/lib/pgsql/16/data
export NEW_DATA=/var/lib/pgsql/17/data
export UPGRADE_DIR=/var/lib/pgsql/upgrade-16-to-17

Adjust them if your current server uses different paths.

1. Inventory the PostgreSQL 16 cluster

Confirm the running version and data directory:

Check the PostgreSQL 16 version and data directory
sudo -u postgres "$OLD_BIN/psql" -XAtc "SELECT version();"
sudo -u postgres "$OLD_BIN/psql" -XAtc "SHOW data_directory;"

Record the cluster encoding, locale, and checksum setting. The new cluster must be initialized compatibly.

Check encoding, locale, and data checksums
sudo -u postgres "$OLD_BIN/psql" -X -c "
SELECT current_setting('server_encoding') AS encoding,
current_setting('lc_collate') AS lc_collate,
current_setting('lc_ctype') AS lc_ctype;
"
sudo -u postgres "$OLD_BIN/pg_controldata" "$OLD_DATA" \
| grep -E "Database cluster state|Data page checksum version"

Check tablespaces because their data also needs space during a copy-mode upgrade:

List PostgreSQL tablespaces
sudo -u postgres "$OLD_BIN/psql" -X -c "
SELECT spcname, pg_tablespace_location(oid)
FROM pg_tablespace
ORDER BY spcname;
"

List extensions in every connectable database:

List extensions in every PostgreSQL database
sudo -iu postgres bash <<'EOF'
while IFS= read -r database; do
echo "=== Database: $database ==="
/usr/pgsql-16/bin/psql -X -d "$database" -c \
"SELECT extname, extversion FROM pg_extension ORDER BY extname;"
done < <(
/usr/pgsql-16/bin/psql -XAtc \
"SELECT datname FROM pg_database WHERE datallowconn AND NOT datistemplate"
)
EOF

Install a PostgreSQL 17-compatible build of every extension containing native code before running pg_upgrade. Do not manually create those extensions in the empty PostgreSQL 17 cluster; pg_upgrade migrates their definitions.

2. Update PostgreSQL 16 to its latest minor release

First update the existing 16.x packages. Plan a short outage for this step.

Update PostgreSQL 16 packages
sudo systemctl stop postgresql-16
sudo dnf upgrade -y 'postgresql16*' 'pgvector_16*'
sudo systemctl start postgresql-16
sudo systemctl status postgresql-16 --no-pager

Verify that the server is ready:

Verify the updated PostgreSQL 16 server
sudo -u postgres "$OLD_BIN/psql" -XAtc "SELECT version();"
sudo -u postgres "$OLD_BIN/pg_isready"

If pgvector was installed from PGXN rather than an RPM, omit pgvector_16* and update it using the same installation method originally used.

3. Take backups

Stop here if there is no tested backup. Create a logical backup of all databases and roles:

Back up all PostgreSQL 16 databases and roles
sudo install -d -m 700 /var/backups/postgresql
sudo -u postgres "$OLD_BIN/pg_dumpall" \
| gzip > "/var/backups/postgresql/pgsql16-before-pg17-$(date +%Y%m%d-%H%M%S).sql.gz"

Back up the configuration files:

Back up the PostgreSQL 16 configuration
sudo install -d -m 700 /root/postgresql16-config-backup
sudo cp -a \
"$OLD_DATA/postgresql.conf" \
"$OLD_DATA/postgresql.auto.conf" \
"$OLD_DATA/pg_hba.conf" \
"$OLD_DATA/pg_ident.conf" \
/root/postgresql16-config-backup/

Check disk space for the data directory and every user tablespace:

Check data size and available disk space
sudo du -sh "$OLD_DATA"
sudo df -h "$OLD_DATA"

A filesystem or VM snapshot taken while PostgreSQL is stopped provides an additional rollback layer.

4. Install PostgreSQL 17 and extensions

Install PostgreSQL 17 alongside PostgreSQL 16:

Install PostgreSQL 17 packages
sudo dnf install -y \
postgresql17-server \
postgresql17-contrib \
postgresql17-devel

If pgvector was installed from the PostgreSQL Yum repository, install its PostgreSQL 17 package:

Install pgvector for PostgreSQL 17
sudo dnf install -y pgvector_17

Prefer the RPM package when available and do not install pgvector with both RPM and PGXN. If an extension is available only through PGXN, select the PostgreSQL 17 pg_config explicitly:

Install a PGXN extension for PostgreSQL 17
sudo pgxnclient install \
--pg_config /usr/pgsql-17/bin/pg_config \
vector

Confirm that both versions are installed:

Check the installed PostgreSQL binaries
"$OLD_BIN/postgres" --version
"$NEW_BIN/postgres" --version
"$NEW_BIN/pg_upgrade" --version

5. Initialize an empty PostgreSQL 17 cluster

Initialize the new cluster with the encoding and locale recorded earlier. Based on the original Rocky Linux installation, these are likely UTF8 and en_US.UTF-8:

Initialize the PostgreSQL 17 cluster
sudo install -d -o postgres -g postgres -m 700 "$NEW_DATA"
sudo -u postgres "$NEW_BIN/initdb" \
--pgdata="$NEW_DATA" \
--encoding=UTF8 \
--locale=en_US.UTF-8

If pg_controldata reported checksum version 1, remove the newly initialized empty directory and initialize it with checksums instead:

Initialize PostgreSQL 17 with data checksums
sudo rm -rf "$NEW_DATA"
sudo install -d -o postgres -g postgres -m 700 "$NEW_DATA"
sudo -u postgres "$NEW_BIN/initdb" \
--pgdata="$NEW_DATA" \
--encoding=UTF8 \
--locale=en_US.UTF-8 \
--data-checksums

Only remove NEW_DATA at this point, while it is a newly initialized empty cluster. Do not start PostgreSQL 17 yet.

6. Stop database clients and PostgreSQL 16

Stop applications, scheduled jobs, backup jobs, monitoring, and connection pools. Review remaining sessions:

Check active PostgreSQL client connections
sudo -u postgres "$OLD_BIN/psql" -X -c "
SELECT pid, usename, datname, application_name, client_addr, state
FROM pg_stat_activity
WHERE pid <> pg_backend_pid()
ORDER BY datname, usename;
"

Stop both database services:

Stop PostgreSQL 16 and PostgreSQL 17
sudo systemctl stop postgresql-16
sudo systemctl stop postgresql-17 2>/dev/null || true
sudo systemctl is-active postgresql-16
sudo systemctl is-active postgresql-17

Both services should report inactive.

7. Run the compatibility check

Create a working directory owned by postgres, then run the PostgreSQL 17 pg_upgrade binary with --check:

Check PostgreSQL 16 and 17 cluster compatibility
sudo install -d -o postgres -g postgres -m 700 "$UPGRADE_DIR"
sudo -iu postgres bash <<'EOF'
cd /var/lib/pgsql/upgrade-16-to-17
/usr/pgsql-17/bin/pg_upgrade \
--check \
--old-bindir=/usr/pgsql-16/bin \
--new-bindir=/usr/pgsql-17/bin \
--old-datadir=/var/lib/pgsql/16/data \
--new-datadir=/var/lib/pgsql/17/data \
--username=postgres
EOF

Do not continue unless the final result is:

Expected pg_upgrade compatibility result
Clusters are compatible

For example, an error about not loading $libdir/vector means the PostgreSQL 17 pgvector library is missing. Install pgvector_17, then rerun the check.

8. Run the PostgreSQL major upgrade

Run the real upgrade in copy mode:

Upgrade PostgreSQL 16 to PostgreSQL 17
sudo -iu postgres bash <<'EOF'
cd /var/lib/pgsql/upgrade-16-to-17
/usr/pgsql-17/bin/pg_upgrade \
--old-bindir=/usr/pgsql-16/bin \
--new-bindir=/usr/pgsql-17/bin \
--old-datadir=/var/lib/pgsql/16/data \
--new-datadir=/var/lib/pgsql/17/data \
--username=postgres \
--jobs="$(nproc)" \
--copy
EOF

Do not use --no-sync on a production server. Read the complete output and run any required extension-update or index-rebuild scripts exactly as printed. Keep the generated old-cluster deletion script, but do not run it yet.

9. Migrate configuration

Do not overwrite PostgreSQL 17's entire postgresql.conf with the PostgreSQL 16 file. Compare the files and manually reapply custom settings that remain valid:

Compare PostgreSQL 16 and 17 configuration files
sudo diff -u \
"$NEW_DATA/postgresql.conf" \
"$OLD_DATA/postgresql.conf" \
| less

Review settings such as listen_addresses, port, memory limits, WAL settings, logging, SSL, and shared_preload_libraries. Any library named in shared_preload_libraries must also be installed for PostgreSQL 17.

The client authentication files can normally be copied after review:

Copy PostgreSQL client authentication configuration
sudo cp -a "$OLD_DATA/pg_hba.conf" "$NEW_DATA/pg_hba.conf"
sudo cp -a "$OLD_DATA/pg_ident.conf" "$NEW_DATA/pg_ident.conf"
sudo chown postgres:postgres \
"$NEW_DATA/pg_hba.conf" \
"$NEW_DATA/pg_ident.conf"

Review postgresql.auto.conf separately and reapply only valid PostgreSQL 17 settings:

Review automatically configured PostgreSQL settings
sudo cat "$OLD_DATA/postgresql.auto.conf"

Check key configuration values before starting the service:

Validate the PostgreSQL 17 configuration
sudo -u postgres "$NEW_BIN/postgres" -D "$NEW_DATA" -C data_directory
sudo -u postgres "$NEW_BIN/postgres" -D "$NEW_DATA" -C listen_addresses

10. Start and verify PostgreSQL 17

Switch the enabled service and start PostgreSQL 17:

Enable and start PostgreSQL 17
sudo systemctl disable postgresql-16
sudo systemctl enable postgresql-17
sudo systemctl start postgresql-17
sudo systemctl status postgresql-17 --no-pager

Verify the server identity, data directory, encoding, and locale:

Verify the upgraded PostgreSQL 17 cluster
sudo -u postgres "$NEW_BIN/psql" -X -c "
SELECT version();
SHOW data_directory;
SHOW server_encoding;
SHOW lc_collate;
SHOW lc_ctype;
"
sudo -u postgres "$NEW_BIN/pg_isready"

Check that all databases are present:

List upgraded PostgreSQL databases and sizes
sudo -u postgres "$NEW_BIN/psql" -X -c "
SELECT datname, pg_size_pretty(pg_database_size(datname)) AS size
FROM pg_database
WHERE datallowconn
ORDER BY datname;
"

11. Update extensions

First run any extension-update script produced by pg_upgrade. Then list the installed extension versions:

Check extensions after the PostgreSQL upgrade
sudo -iu postgres bash <<'EOF'
while IFS= read -r database; do
echo "=== Database: $database ==="
/usr/pgsql-17/bin/psql -X -d "$database" -c \
"SELECT extname, extversion FROM pg_extension ORDER BY extname;"
done < <(
/usr/pgsql-17/bin/psql -XAtc \
"SELECT datname FROM pg_database WHERE datallowconn AND NOT datistemplate"
)
EOF

Update pgvector in each database that uses it:

Update and verify the pgvector extension
sudo -u postgres "$NEW_BIN/psql" \
--dbname=your_database \
--command="ALTER EXTENSION vector UPDATE;"
sudo -u postgres "$NEW_BIN/psql" \
--dbname=your_database \
--command="SELECT extversion FROM pg_extension WHERE extname = 'vector';"

Update other extensions only where they are installed:

Update PostgreSQL contrib extensions
ALTER EXTENSION pg_trgm UPDATE;
ALTER EXTENSION btree_gist UPDATE;

12. Regenerate optimizer statistics

pg_upgrade does not transfer optimizer statistics. Regenerate them in stages before returning the server to normal traffic:

Regenerate PostgreSQL optimizer statistics in stages
sudo -u postgres "$NEW_BIN/vacuumdb" \
--all \
--analyze-in-stages \
--jobs="$(nproc)"

After normal operation resumes, run a complete analyze:

Run a complete PostgreSQL analyze
sudo -u postgres "$NEW_BIN/vacuumdb" \
--all \
--analyze-only \
--jobs="$(nproc)"

13. Validate before deleting PostgreSQL 16

Review the service log:

Check recent PostgreSQL 17 service logs
sudo journalctl \
-u postgresql-17 \
--since "1 hour ago" \
--no-pager

Check for invalid indexes in each important database:

Find invalid PostgreSQL indexes
sudo -u postgres "$NEW_BIN/psql" -X -d your_database -c "
SELECT n.nspname AS schema_name, c.relname AS index_name
FROM pg_index i
JOIN pg_class c ON c.oid = i.indexrelid
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE NOT i.indisvalid
ORDER BY 1, 2;
"

Validate database and table counts, important row counts, application login, reads and writes, scheduled jobs, backups, monitoring, connection pools, and pgvector searches and indexes. Keep the PostgreSQL 16 packages and /var/lib/pgsql/16/data for an agreed retention period.

Roll back before production writes

Because the upgrade used --copy, PostgreSQL 16 remains unchanged. Before PostgreSQL 17 receives production writes, rollback is straightforward:

Roll back to PostgreSQL 16 before new writes
sudo systemctl stop postgresql-17
sudo systemctl disable postgresql-17
sudo systemctl enable postgresql-16
sudo systemctl start postgresql-16
sudo systemctl status postgresql-16 --no-pager

After clients write to PostgreSQL 17, restarting PostgreSQL 16 would discard those new writes. Stop all clients and plan how to export or reconcile the PostgreSQL 17 changes before attempting rollback.

Final cleanup

Only after PostgreSQL 17 has passed validation and fresh backups are working, disable PostgreSQL 16 permanently:

Disable the old PostgreSQL 16 service
sudo systemctl disable postgresql-16

After the retention period, remove the old packages. Review the DNF transaction before accepting it so that libraries required by other software are not removed:

Remove the old PostgreSQL 16 packages
sudo dnf remove \
postgresql16-server \
postgresql16-contrib \
postgresql16-devel

Finally, delete the old cluster only when you are certain it is no longer needed. Use the deletion script generated by pg_upgrade, or archive the old data directory according to your backup and retention policy.

Installing Snell Server on Rocky Linux 9 With Network Bandwidth Limiting

Permalink

My previous Snell installation note was written for CentOS 7 in 2020. Several years later, CentOS 7 is obsolete, predictable network interface names are common, and Snell has moved to version 5.

This guide installs Snell Server on Rocky Linux 9, manages it with systemd, opens the required firewall ports, and optionally limits the server’s outbound network throughput using Linux Traffic Control.

Snell is a lightweight encrypted proxy protocol developed for Surge. The official server is distributed as a single binary with no external runtime dependencies other than glibc. Snell v5 also adds a QUIC proxy mode, which requires the server’s UDP port to be reachable.

1. Update Rocky Linux and install the required tools

Log in as root, or prefix the commands with sudo.

Update Rocky Linux and install prerequisites
dnf update -y
dnf install -y wget unzip firewalld iproute-tc

The iproute-tc package provides the tc command used later for bandwidth limiting.

Enable and start the firewall:

Enable and start firewalld
systemctl enable --now firewalld
systemctl status firewalld

Rocky Linux 9 uses firewalld for common firewall management. Its runtime and permanent configurations are separate, so permanent rules must be explicitly added when they should survive a reboot.

2. Create a dedicated Snell account

Snell does not need an interactive login account:

Create the non-login Snell service account
useradd \
--system \
--home-dir /opt/snell \
--create-home \
--shell /sbin/nologin \
snell

Create the application directory:

Create and secure the Snell application directory
mkdir -p /opt/snell
chown snell:snell /opt/snell
chmod 750 /opt/snell

3. Download Snell Server

At the time of writing, the latest official release is Snell Server 5.0.1.

For a typical x86-64 VPS:

Download Snell Server
cd /tmp
wget https://dl.nssurge.com/snell/snell-server-v5.0.1-linux-amd64.zip
unzip snell-server-v5.0.1-linux-amd64.zip
install \
-o snell \
-g snell \
-m 750 \
snell-server \
/opt/snell/snell-server
rm -f snell-server snell-server-v5.0.1-linux-amd64.zip

For an ARM64 server, use the official AArch64 package instead:

Official Snell Server download for ARM64
https://dl.nssurge.com/snell/snell-server-v5.0.1-linux-aarch64.zip

The official Snell page currently provides builds for AMD64, i386, AArch64, and ARMv7.

Confirm that the binary runs:

Confirm that the binary runs
/opt/snell/snell-server --help

4. Generate the Snell configuration

Run the built-in configuration wizard as the snell user:

Run the Snell configuration wizard
cd /opt/snell
sudo -u snell ./snell-server

The wizard asks for values such as:

  • Listening address
  • Listening port
  • Pre-shared key, or PSK
  • IPv6 support

For example, select port 11666.

After the wizard finishes, verify the generated file:

Inspect the generated Snell configuration
cat /opt/snell/snell-server.conf

A typical configuration resembles:

Example Snell Server configuration
[snell-server]
listen = 0.0.0.0:11666
psk = YOUR_RANDOM_SECRET
ipv6 = false

Protect the configuration because it contains the PSK:

Restrict access to the Snell configuration
chown snell:snell /opt/snell/snell-server.conf
chmod 600 /opt/snell/snell-server.conf

Do not publish the real PSK in screenshots, shell history, or a blog post.

5. Create the systemd service

Create /etc/systemd/system/snell.service:

Create the Snell systemd service file
cat >/etc/systemd/system/snell.service <<'EOF'
[Unit]
Description=Snell Proxy Service
Documentation=https://kb.nssurge.com/surge-knowledge-base/release-notes/snell
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=snell
Group=snell
WorkingDirectory=/opt/snell
ExecStart=/opt/snell/snell-server -c /opt/snell/snell-server.conf
Restart=on-failure
RestartSec=5s
LimitNOFILE=32768
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths=/opt/snell
[Install]
WantedBy=multi-user.target
EOF

Reload systemd and start Snell:

Reload systemd and start Snell
systemctl daemon-reload
systemctl enable --now snell

Check its status:

Check Snell service status
systemctl status snell --no-pager

View recent logs:

View recent Snell service logs
journalctl -u snell -n 100 --no-pager

Follow the logs in real time:

Follow Snell service logs
journalctl -u snell -f

Confirm that the selected port is listening:

Confirm that the Snell port is listening
ss -lntup | grep 11666

6. Open the Snell port in firewalld

Snell accepts its primary connection over TCP. Snell v5’s QUIC proxy mode additionally uses UDP, so open the same port for both protocols.

Open the Snell TCP and UDP ports
firewall-cmd --permanent --zone=public --add-port=11666/tcp
firewall-cmd --permanent --zone=public --add-port=11666/udp
firewall-cmd --reload

Verify the rules:

List the configured firewalld ports
firewall-cmd --zone=public --list-ports

Expected output should include:

Expected firewalld port listing
11666/tcp 11666/udp

If the server uses a cloud-provider firewall or security group, the same TCP and UDP ports must also be opened there.

7. Configure Surge

Add a proxy entry to the Surge profile:

Configure Surge
[Proxy]
My-Snell-Server = snell, YOUR_SERVER_IP, 11666, psk=YOUR_RANDOM_SECRET, version=5

Replace:

  • YOUR_SERVER_IP with the VPS public IP address.
  • YOUR_RANDOM_SECRET with the PSK from snell-server.conf.

The Snell v5 server remains backward compatible with v4 clients. However, QUIC proxy mode is a v5 feature.

8. Limit outbound network throughput

Some VPS providers charge for excess bandwidth or impose fair-use limits. Linux Traffic Control can apply a maximum outbound rate to a network interface.

Linux tc manages queueing disciplines that schedule packets as they leave an interface. The Token Bucket Filter, or TBF, is suitable for applying a simple maximum transmission rate.

Find the public network interface

Do not assume that the interface is named eth0. Rocky Linux systems frequently use names such as:

  • ens3
  • ens18
  • enp1s0
  • enp0s3

Find the interface used by the default route:

Show the default-route network interface
ip route show default

For example:

Example default-route output
default via 192.0.2.1 dev eth0 proto static metric 100

In this example, the interface is eth0.

You can extract it directly with:

Extract the default-route interface name
ip route show default | awk '{print $5; exit}'

Rocky Linux recommends modern tools such as ip and nmcli for network inspection and configuration.

Test the rule manually

The following command limits outbound traffic on eth0 to approximately 36 Mbit/s:

Apply a 36 Mbit/s outbound TBF limit
tc qdisc replace dev eth0 root tbf \
rate 36mbit \
burst 128kbit \
latency 200ms

Check the active queueing discipline:

Inspect the active TBF queueing discipline
tc -s qdisc show dev eth0

Remove the rule:

Remove the temporary TBF limit
tc qdisc del dev eth0 root

The limit applies to traffic transmitted through eth0. It therefore affects Snell, SSH, package downloads, web servers, and any other outbound service using that interface.

It does not directly limit inbound traffic. TCP downloads may nevertheless slow down indirectly because acknowledgements and response traffic leave through the rate-limited interface.

Persist the limit with systemd

Create /etc/systemd/system/netlimit.service:

Create the netlimit systemd service file
cat >/etc/systemd/system/netlimit.service <<'EOF'
[Unit]
Description=Limit outbound network throughput
After=network-online.target
Wants=network-online.target
Before=snell.service
[Service]
Type=oneshot
ExecStart=/usr/sbin/tc qdisc replace dev eth0 root tbf rate 36mbit burst 128kbit latency 200ms
ExecStop=-/usr/sbin/tc qdisc del dev eth0 root
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
EOF

Replace eth0 with the actual public network interface found earlier.

The use of replace, rather than add, makes the service more tolerant of an existing root queueing discipline. The leading - in ExecStop tells systemd not to treat a missing queueing discipline as a fatal stop error.

Enable and start the service:

Reload systemd and start netlimit
systemctl daemon-reload
systemctl enable --now netlimit

Check its status:

Check netlimit service status
systemctl status netlimit --no-pager

Verify the active rate:

Verify the configured TBF rate
tc -s qdisc show dev eth0

Example output:

Example TBF queueing-discipline output
qdisc tbf 8001: root refcnt 2 rate 36Mbit burst 128Kb lat 200ms

Restarting the service reapplies the configured rate:

Restart netlimit to reapply the rate
systemctl restart netlimit

To disable bandwidth limiting:

Disable the persistent bandwidth limit
systemctl disable --now netlimit

Then confirm that the TBF rule is gone:

Confirm that the TBF rule is removed
tc qdisc show dev eth0

9. Testing the bandwidth limit

The most reliable test is to transfer a sufficiently large file from the Snell server or run a speed test from a remote client.

Remember the unit conversion:

Convert the configured Mbit/s rate to MB/s
36 Mbit/s ÷ 8 = approximately 4.5 MB/s

Protocol overhead means that the observed application-level transfer speed will normally be slightly lower than 4.5 MB/s.

You can watch the TBF counters while testing:

Watch TBF counters while testing
watch -n 1 'tc -s qdisc show dev eth0'

Pay attention to:

  • Sent bytes
  • Sent packets
  • Dropped packets
  • Overlimits
  • Backlog

An increasing overlimits counter is normal: it shows that TBF is delaying packets to enforce the configured rate. A large number of dropped packets or a continuously growing backlog may indicate that the burst or latency values need adjustment.

10. Maintenance commands

Restart Snell:

Restart the Snell service
systemctl restart snell

Check Snell:

Check Snell service status during maintenance
systemctl status snell --no-pager

View logs:

View Snell logs since today
journalctl -u snell --since today

Review the configuration:

Inspect the Snell configuration
sudo -u snell cat /opt/snell/snell-server.conf

Check the firewall:

List the configured firewall ports
firewall-cmd --zone=public --list-ports

Check bandwidth limiting:

Inspect the active bandwidth-limit rule
tc -s qdisc show dev eth0

Check both services after a reboot:

Check Snell and netlimit after reboot
systemctl is-active snell netlimit
systemctl is-enabled snell netlimit

11. Upgrading Snell Server

Download and extract the newer binary into a temporary directory, then stop the service and replace the existing executable:

Upgrading Snell Server
systemctl stop snell
install \
-o snell \
-g snell \
-m 750 \
/path/to/new/snell-server \
/opt/snell/snell-server
systemctl start snell
systemctl status snell --no-pager

The configuration file can normally remain in place, but release notes should be reviewed before each upgrade.

Conclusion

Compared with the old CentOS 7 setup, the Rocky Linux 9 version is mostly familiar:

  • Snell still runs as a small standalone binary.
  • systemd manages startup and recovery.
  • firewalld exposes the selected TCP and UDP ports.
  • tc and TBF provide a simple outbound bandwidth ceiling.

The main detail to watch is the network interface name. Copying eth0 blindly may cause netlimit.service to fail on servers whose public interface is named ens3, ens18, or enp1s0.

Finally, remember that the simple TBF rule limits the entire network interface. It is suitable for a dedicated Snell VPS. On a shared server, more advanced tc classes and filters would be required to limit only Snell traffic.

Fixing Bun Run Dev Failure in Opencode Desktop

Permalink

Today I encountered a classic "works on my machine... wait, no it doesn't" moment while setting up the Opencode Desktop project. Running bun run dev failed with a cryptic Error: Electron uninstall message. Here's the full story of how I diagnosed and fixed it.

My 2026 Monthly Subscription Review

Permalink

Like the previous years 2022 / 2023 / 2024 and 2025, here is my 2026 subscription list. The first number is RMB per month.

My pay:

  1. (68) iCloud 2T
  2. (17) Apple Music family plan
  3. (24) Dragonruby Pro (annual 42 USD)
  4. (33) Bandwagon host (monthly 33 USD, shared)
  5. (6) Adblock Pro (annual 70 RMB)
  6. (6.5) This blog domain (annual 11 USD)
  7. (1.5) 香哈菜谱 (annual 18 RMB)
  8. (49) AWS hosting (3 years 206 USD, monthly 1.35 USD)
  9. (16.8) Meituan biking (monthly)
  10. (110) Cursor.sh AI editor. (yearly 192 USD)
  11. (8.2) Ivory for Mastodon (yearly 98 RMB)
  12. (8) IndieWeb.Social Backer. (monthly 1.5 SGD)
  13. (31) Sublime Text and Merge (3 years, 152 AUD)
  14. (26) Surge (yearly, 46 USD)
  15. (156) ChatGPT (monthly, 22 USD)
  16. (17.9) Kapeli Dash.app (yearly, 215.33 RMB)
  17. (199) Kimi Allegretto for code (monthly, 199 RMB)

TH pay:

  1. (TH) OpenCode Go and Zen (depend on bill)
  2. (TH) Gemini & Nano Banana (depend on bill)
  3. (52.25) Google Workspace Business Starter (yearly 92.65 USD)

So totally 777.9 RMB per month to pay. In the previous 2025 year it was 773 RMB, so almost same compared to 2025.

Build X86 Electron on Arm64 Mac

Permalink

When building a Windows Electron app on Apple Silicon macOS, electron-builder --win defaults to the host architecture — so running:

Electron-builder default architecture on Apple Silicon
cd packages/desktop && bun run package:win

produces arm64 Windows binaries instead of the expected x64.

The Fix

Force x64 explicitly with the --x64 flag when invoking the package command:

Build the Windows package explicitly for x64
cd /path/to/packages/desktop
bun run build
bun run package:win -- --x64 --publish never

Or call electron-builder directly:

Build the x64 Windows package with electron-builder
npx electron-builder --win --x64 --publish never --config electron-builder.config.ts

This produces the standard NSIS x64 artifacts:

Expected x64 NSIS package artifacts
dist/SigmaAgents-win-x64-<version>.exe
dist/SigmaAgents-win-x64-<version>.exe.blockmap
dist/latest.yml

To pin a specific version (e.g. 1.14.42), set the OPENCODE_VERSION env var before the build:

Build a specific x64 package version
OPENCODE_VERSION=1.14.42 bun ./scripts/prepare.ts
bun run build
bun run package:win -- --x64 --publish never

Follow-up: Native Module Mismatch

After building with --x64, the .exe may launch but crash with:

Follow-up: Native Module Mismatch
Error: Cannot find module './windowsTerminal'

Root cause: bun run build resolves native modules (e.g. @lydell/node-pty) based on the host platform and architecture — darwin-arm64 — rather than the target win32-x64. The bundle ends up importing the macOS native binding, which fails when the Windows .exe tries to load it at runtime.

The Fix

Set RUST_TARGET=x86_64-pc-windows-msvc so the build step selects the correct native bindings for the target platform:

Set the Rust target for Windows x64 bindings
cd /path/to/packages/desktop
RUST_TARGET=x86_64-pc-windows-msvc bun run build
RUST_TARGET=x86_64-pc-windows-msvc bun run package:win -- --x64 --publish never

Verify

After the build, confirm the main bundle imports the correct platform binding:

Check the bundled node-pty platform binding
rg "node-pty" out/main/index.js
# should show: @lydell/node-pty-win32-x64

And check the packaged app.asar includes the right native module:

Check the packaged node-pty native module
npx asar list dist/win-unpacked/resources/app.asar | rg "windowsTerminal"
# should exist under: node_modules/@lydell/node-pty-win32-x64/lib/

TL;DR for cross-building on macOS

Both env vars are needed together for a working Windows build:

TL;DR for cross-building on macOS
RUST_TARGET=x86_64-pc-windows-msvc \
bun run build && \
bun run package:win -- --x64 --publish never

Without RUST_TARGET, you get a working .exe installer but a broken main process that can't find its native modules.

Caveat: Code Signing

The .exe built on macOS will not be Windows-code-signed. Electron-builder's signing routines typically gate on process.platform === "win32" (and often GITHUB_ACTIONS === "true"), so they intentionally skip signing on macOS. If you need a signed Windows release artifact, you must either build and sign on a Windows machine or add a macOS-compatible signing path to your config.

How to Fix Bun v1.3.12 Based Opencode Got Killed: 9 After Bun Turbo Build

Permalink

New bun 1.3.12 release but I got:

New bun 1.3.12 release but I got:
$ ./opencode --version
Killed: 9

In fact, the binary was fine. The code signature was not.

Diagnosis

A quick check with codesign revealed the real culprit:

Check the binary code signature
$ codesign -vv ./opencode
./opencode: invalid or unsupported format for signature
In architecture: arm64

On Apple Silicon, macOS enforces code signing strictly. A malformed or corrupt LC_CODE_SIGNATURE load command will cause the kernel to terminate the process immediately with SIGKILL (Killed: 9)—before a single line of your code runs.

You can confirm the signature load command exists with otool:

Confirm the code-signature load command
$ otool -l ./opencode | grep LC_CODE_SIGNATURE
cmd LC_CODE_SIGNATURE

The Fix

Remove the broken signature and replace it with a local ad-hoc signature:

Replace the broken code signature
codesign --remove-signature ./opencode
codesign --sign - --force ./opencode

Then verify:

Verify the repaired code signature
$ codesign -vv ./opencode
./opencode: valid on disk
./opencode: satisfies its Designated Requirement
$ ./opencode --version
0.0.0-eric_dev-202604120639

Using Git blame.ignoreRevsFile Options to Ignore Your Auto Format Coding Commits

Permalink

After running any auto-format on a repository, git blame becomes almost useless because every line is “last touched” by the formatting commit.

Git has a built-in solution: tell git blame to ignore specific commits.

Create .git-blame-ignore-revs in root repo.

.git-blame-ignore-revs
# Formatting / reindent / whitespace-only commits
1a2b3c4d
abcdef01

Enable it in your repo via git

Enable it in your repo via git
git config --local blame.ignoreRevsFile .git-blame-ignore-revs

Now when you run git blame, those commits will be skipped and you can see the real history again.

Notes

  • Commit .git-blame-ignore-revs into your repo so GitHub (and other developers) can use the same ignore list.
  • GitHub also reads .git-blame-ignore-revs automatically in the web UI, so you’ll get the same “clean blame” there too.