Configuration change control

Production runbook for planned configuration changes on Pixop Prime (AWS Marketplace guest or KVM QCOW2 appliance). Pair with the control-plane map in Configuration. This document does not replace WAN firewall design (Network and transport) or transport fault isolation (Transport troubleshooting).

Version gate: Steps below may include relay-only validation (tsduckctl transport, bonded URI groups). Run pixop-version and skip relay workflows unless command -v tsduck-transport-info succeeds and your installed templates use TRANSPORT_BACKEND=relay. Features are capability-gated—not implied by a Marketplace product version label.

1. Change record and roles

Before any production edit, open a maintenance or change ticket that survives handover between NOC shifts.

Roles: The change owner edits configuration and executes restarts. The verifier runs read-only checks and downstream decode/viewing tests. Pixop support owns changes to healthcheck.env, systemd units, and qualified media-path flags outside marked operator blocks in nvencc.args.

2. Classify the change

Match your work to the smallest restart surface. Combining unrelated classes in one window complicates rollback.

Native vs relay: On native SRT appliances, transport class D/E uses .env SRT keys (TRANSPORT_BACKEND=native or unset). On relay-capable appliances, default templates may use TRANSPORT_BACKEND=relay with *.ingest.args / *.egress.args. Setting relay without relay helpers fails at start—confirm binaries and templates before switching backends (Transport configuration).

--apply helpers: tsduckctl detect-video-pid main --apply and tsduckctl tune-latency main --apply rewrite ingress .env and restart main—treat as class D.

Pixop does not certify unverified “zero outage” configuration changes on a single encoder appliance.

3. Backup and restore strategy

What to back up privately

Store backups on the appliance or jump host in a root-only directory. Do not email tarballs containing passphrases or JWTs.

What packers are (and are not)

• pixop-pack-config — Sanitized copy of /etc/pixop-live with redaction; no license.jwt / Pulse / Docker bodies; does not capture params.json (lives under /var/lib/pixop-live/parameters/).
• pixop-support-bundle — Full diagnostic archive; nests pack-config; still not a production restore source for secrets or params.json.

Private backup (example)

Run as root on the guest (sudo -i or root shell). Use pixopctl / tsduckctl without extra sudo — the wrappers self-elevate when needed. Reading /etc/pixop-live and writing under /root requires root. Replace CHG-12345 with your ticket id. Record paths and hashes in the change ticket, not file contents.

BKROOT="/root/pixop-config-backup/CHG-12345-$(date -u +%Y%m%dT%H%M%SZ)"
install -d -m 0700 "$BKROOT"
[[ -f /etc/pixop-live/nvencc.args ]] && cp -a /etc/pixop-live/nvencc.args "$BKROOT/"
[[ -f /etc/pixop-live/network.env ]] && cp -a /etc/pixop-live/network.env "$BKROOT/"
[[ -d /etc/pixop-live/tsduck ]] && cp -a /etc/pixop-live/tsduck "$BKROOT/"
if [[ -f /var/lib/pixop-live/parameters/params.json ]]; then
  cp -a /var/lib/pixop-live/parameters/params.json "$BKROOT/"
fi
find "$BKROOT" -type f ! -name SHA256SUMS -print0 | sort -z | xargs -0 -r sha256sum > "$BKROOT/SHA256SUMS"
chmod -R u=rwX,go= "$BKROOT"

Restore (example)

Only after stopping the change window and notifying downstream. Run as root on the guest. The block below only restores files; restarts are separate so a single-file rollback does not restart the whole stack.

BKROOT="/root/pixop-config-backup/CHG-12345-20261021T120000Z"   # your path
sha256sum -c "$BKROOT/SHA256SUMS"

[[ -f "$BKROOT/nvencc.args" ]] && cp -a "$BKROOT/nvencc.args" /etc/pixop-live/nvencc.args
[[ -f "$BKROOT/network.env" ]] && cp -a "$BKROOT/network.env" /etc/pixop-live/network.env
[[ -d "$BKROOT/tsduck" ]] && cp -a "$BKROOT/tsduck/." /etc/pixop-live/tsduck/
if [[ -f "$BKROOT/params.json" ]]; then
  install -d -m 2750 -o 1001 -g pixop /var/lib/pixop-live/parameters
  cp -a "$BKROOT/params.json" /var/lib/pixop-live/parameters/params.json
  chown 1001:pixop /var/lib/pixop-live/parameters/params.json
  chmod 0640 /var/lib/pixop-live/parameters/params.json
fi

Restart only what you restored (match §2 class):

After transport restores, run tsduckctl credentials locally to verify the credential sheet—do not print passphrases into tickets or log collectors.

Image rollback: Use pixop-image-update rollback live|tsduck only when LIVE_IMAGE_PREV / TSDUCK_IMAGE_PREV in env files points at a locally present digest (pixop-image-update).

4. Dry-run and on-air acceptance

Pre-change baseline (read-only)

pixop-version
pixop-preflight
pixop-stack status
tsduckctl check main
tsduckctl check egress-main
pixopctl health
pixop-flow --secs 5

When command -v tsduck-transport-info succeeds:

tsduckctl transport main
tsduckctl transport egress-main

Interpretation:

For SRT, tsduckctl check may report listener bound while the caller peer is offline—that is expected in listener mode.

Post-change acceptance

Repeat baseline commands. With active contribution:

1. Confirm pixop-flow shows motion on Ingress -> Pixop, Pixop -> replacement, and Final merged TS / egress stages as applicable.
2. Verify downstream decoder: correct program, resolution, chroma, and HDR/SDR signalling vs your ticket.
3. For class C metadata changes, remember VUI flags in nvencc.args set signalling; actual tone mapping may still require UI/params.json alignment (Live output and encoding, Picture processing).

Rollback by class

Emergency: Return to last known-good backup under a named ticket—avoid “factory reset” or copying another customer’s /etc tree. pixopctl recover is for wedged pipelines, not routine rollback (Monitoring and health signals).

5. Security and operational hazards

• No inline secrets in tickets, chat, or runbook copies: JWT, SRT passphrases, Pulse passwords, Docker Hub tokens.
• Restrict TCP 8080 to management CIDRs; UI and metrics are HTTP without TLS on the guest.
• Review pixop-pack-config / support bundle tarballs before upload; redaction is best-effort.
• PIXOP_HEALTHCHECK_RESTART=1 (default in healthcheck.env) can schedule pixopctl recover during prolonged unhealthy metrics—account for that in change timelines.
• Heavy pixop-process file jobs contend for GPU with live enhancement; avoid batch loads during the same window as class C or picture changes.

6. Worked examples (no secrets)

Example A — Output HDR signalling to SDR VUI (class C)

Goal: Downstream chain expects BT.709 SDR signalling on the enhanced HEVC elementary stream.

1. Back up nvencc.args (§3).
2. In the operator colour block, disable the active HLG group and enable the commented SDR BT.709 group per Live output and encoding—one complete profile only.
3. pixopctl restart.
4. Accept: downstream VUI reports BT.709; verify picture appearance separately—if the source was HDR, you may still need UI tone-mapping / ITM changes (class A/B), not encoder flags alone.

Example B — Ingress video PID realignment (class D)

Goal: Replacement merge targets the correct video PID after a source lineup change.

1. With stable inbound MPEG-TS, run read-only: tsduckctl detect-video-pid main --secs 10 (no --apply).
2. If the result is unambiguous, run tsduckctl detect-video-pid main --secs 10 --apply (updates ORIGINAL_VIDEO_PID / INPUT_PCR_PID and restarts ingress).
3. Accept: pixop-flow shows replacement traffic; downstream program video returns; re-check audio PIDs if your workflow carries separate audio rules.

Example C — Pipeline latency tune (class D)

1. tsduckctl tune-latency main (report only).
2. If recommendation differs from PIPELINE_LATENCY in main.env, apply during a negotiated window: tsduckctl tune-latency main --apply.
3. Accept: end-to-end latency within agreed bound; no elevated pixop-flow exit 3 (backbone bitrate pressure against TS_TARGET_BITRATE).

Related

• Configuration
• Live output and encoding
• Picture processing
• Transport configuration
• Network and transport
• Monitoring and troubleshooting
• pixop-pack-config · pixop-support-bundle