jr-public/jrich-wakeword-trainer
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity

0.2.1-jrich-3 cannot complete setup or training on any host: container uid/permission model + archive-layout import bug #1

New Issue
Closed
opened 2026-08-14 12:38:39 -05:00 by westtexasfish · 4 comments
westtexasfish commented 2026-08-14 12:38:39 -05:00
Owner
Copy Link
Copy Source

Summary

Release 0.2.1-jrich-3 cannot complete asset setup or start training on a correctly configured host. Four separate blocking defects, plus one silent-corruption defect that would produce a bad model once the blockers are cleared.

None of this is host misconfiguration and none of it is WSL-specific - a plain Linux host with the default umask 022 fails identically. Every finding below was reproduced, bisected, and then re-verified against a patched copy that trains end to end.

Environment used for diagnosis: Ubuntu 24.04 on WSL2, Python 3.12.9, Docker 29.7.2, RTX 3090 (driver 591.86), NVIDIA Container Toolkit working, 855 GB free. All 28 files verified against the shipped SHA256SUMS.

1. Container runs as uid 0 with --cap-drop ALL against host-owned bind mounts - BLOCKER

scripts/wakeword_trainer.py passes --cap-drop ALL. The image declares no USER, so the container is uid 0, but --cap-drop ALL removes CAP_DAC_OVERRIDE. Root cannot write a bind-mounted directory it does not own, and the launcher creates the cache as the invoking user at mode 0755:

PermissionError: [Errno 13] Permission denied: '/data/piper'
error: LiveKit asset setup failed

Bisected the hardening flags individually - --read-only and no-new-privileges are both harmless here; --cap-drop ALL is the sole cause.

The same pattern applies to the training stage's /run mount in scripts/wakeword_training.py. Simulated it directly:

mkdir: cannot create directory '/run/output': Permission denied
touch: cannot touch '/run/pipeline-time-v.txt': Permission denied

2. --read-only rootfs + HOME=/root with no HF_HOME - BLOCKER

The container gets --tmpfs /tmp but HOME stays /root on the read-only rootfs, and no HuggingFace cache variables are set. Every HF-backed dataset fails:

WARNING  Failed to download ACAV100M features: I/O error: Read-only file system (os error 30)
WARNING  Failed to download validation features: ...
WARNING  Failed to download RIRs: ...
WARNING  Failed to download MUSAN noise: ...
INFO     Setup complete!

After a "successful" setup the cache held 167 MB instead of ~17 GB - only the Piper checkpoint, which uses plain HTTP rather than huggingface_hub.

3. Root-created 0600 files break the host-side inventory - BLOCKER

Even when writes succeed, the container creates local_dir/.cache/huggingface/trees/*.json as root:root mode 0600. The host-side inventory then cannot read them:

error: [Errno 13] Permission denied:
  .../livekit-0.2.1/backgrounds/.cache/huggingface/trees/3edcf...json

Real exit code 2. The resulting cache also could not be removed without going through a root container.

4. /opt/jr-wakeword/* is 0600 root:root in the image - BLOCKER for the fix

Fixing 1-3 by running the container as the invoking user immediately hits:

python: can't open file '/opt/jr-wakeword/run-local-experiment.py': [Errno 13] Permission denied

Every file the Dockerfile COPYs is mode 0600:

600 /opt/jr-wakeword/run-local-experiment.py
600 /opt/jr-wakeword/run-commercial-pipeline.py
600 /opt/jr-wakeword/wakeword_training_image.py
600 /opt/jr-wakeword/finalize-run.py
600 /opt/jr-wakeword/stage_evidence.py
...

COPY preserves build-context modes, and nothing in the Dockerfile loosens them. This was invisible while the container ran as root. /opt/conda is 755, so setup alone survives an unprivileged uid; only the training stage breaks.

This means the permission fix cannot be launcher-only - the image must be rebuilt.

5. Archive-layout import bug - BLOCKER, independent of permissions

scripts/wakeword_training.py does a bare:

from scripts.wakeword_training_image import build_stage_command

The entry point scripts/jrich-wakeword-trainer puts scripts/ on sys.path, not the archive root, so the scripts. prefix is unresolvable in an extracted archive. Training dies before any container launches:

terminal_reason : executor_error:ModuleNotFoundError
stage           : pipeline
elapsed         : 0.048s

The launcher already solves this exact problem in scripts/wakeword_trainer.py:

# Import shape differs only between the extracted
# archive (sibling module) and repository tests (scripts package).
try:
    from wakeword_training import RunKind, WakewordTrainingRequest
except ModuleNotFoundError:
    from scripts.wakeword_training import RunKind, WakewordTrainingRequest

There are exactly two scripts.-qualified imports in the archive; one is guarded, one is not. Training therefore only ever worked from a repo checkout - the release was never exercised from its own artifact.

6. Failed downloads reported as success - SILENT CORRUPTION, not a blocker

This one survives all the fixes above and is arguably the most dangerous.

livekit/wakeword cli.py downgrades every download failure to a WARNING, then prints Setup complete! and exits 0. The runner seals whatever landed into the immutable source lock and treats it as a valid input set. Observed: a source lock over 1008 files / 17.8 GB in which the MUSAN background corpus was 227 of 774 files, with nothing in the provenance record indicating a problem.

Compounding it, _download_musan_noise short-circuits on any non-zero count:

existing = list(bg_dir.glob("**/*.wav"))
if existing:
    logger.info(f"Background noise already present: {len(existing)} files in {bg_dir}")
    return

So one transient HTTP 429 permanently pins the corpus at a partial count, and every subsequent setup reports success. Recovery requires deleting the entire backgrounds/ directory and re-pulling all 1.1 GB. Reproduced twice (227/774, then 768/774); a wipe-and-retry loop was needed to reach 774/774.

There is also no HF_TOKEN passthrough and no retry on 429. Unauthenticated pulls rate-limit reliably at this data volume:

WARNING  Warning: You are sending unauthenticated requests to the HF Hub.
         Please set a HF_TOKEN to enable higher rate limits and faster downloads.

Suggested: treat an incomplete asset set as fatal, verify counts against the expected manifest before sealing a source lock, and add token passthrough with retry/backoff.

7. Windows entry point is blocked by execution policy - end-user papercut

Under the default RemoteSigned policy for CurrentUser, a downloaded start-windows.ps1 carries mark-of-the-web and "Run with PowerShell" fails before anything starts:

File ...\start-windows.ps1 cannot be loaded. The file is not digitally signed.
You cannot run this script on the current system.

Users can Unblock-File, but the README says to right-click -> Run with PowerShell, which does not work on a stock machine. Consider signing the script, shipping a .cmd shim, or documenting the unblock step.

Proposed fix

Launcher (this repo)

Applies cleanly to a pristine 0.2.1-jrich-3 archive with patch -p1:

--- a/scripts/wakeword_trainer.py
+++ b/scripts/wakeword_trainer.py
@@ -83,6 +83,8 @@
 ) -> tuple[str, ...]:
     """Build the explicit networked cache-population command; training stays offline."""
 
+    from wakeword_training import unprivileged_container_args
+
     cache_dir.mkdir(parents=True, exist_ok=True)
     return (
         "docker",
@@ -96,6 +98,7 @@
         "ALL",
         "--security-opt",
         "no-new-privileges",
+        *unprivileged_container_args(),
         "--mount",
         f"type=bind,src={cache_dir.resolve()},dst=/data",
         "--mount",
@@ -352,6 +355,8 @@
 
 
 def _preflight(plan: dict[str, Any]) -> None:
+    from wakeword_training import unprivileged_container_args
+
     runtime = plan["runtime"]
     for executable in ("docker", "nvidia-smi"):
         if shutil.which(executable) is None:
@@ -384,6 +389,9 @@
             "ALL",
             "--security-opt",
             "no-new-privileges",
+            # Preflight under the same uid the real stages use, so this check
+            # cannot pass in a configuration training will not run in.
+            *unprivileged_container_args(),
             runtime["image_ref"],
             "python",
             "-c",
--- a/scripts/wakeword_training.py
+++ b/scripts/wakeword_training.py
@@ -73,6 +73,32 @@
 )
 
 
+def unprivileged_container_args() -> tuple[str, ...]:
+    """Run the container as the invoking user with caches off the read-only rootfs.
+
+    The image declares no ``USER``, so without this the container is uid 0 while
+    ``--cap-drop ALL`` removes ``CAP_DAC_OVERRIDE``.  Root then cannot write a
+    bind-mounted host directory it does not own, and any file it does create is
+    unreadable to the host user that has to inventory it afterwards.  Matching the
+    host uid/gid keeps both directions working under the same dropped capabilities.
+
+    ``HOME`` defaults to ``/root`` on the read-only rootfs, so HuggingFace and
+    friends fail with ``Read-only file system``.  Point the HOME-derived cache
+    paths at the writable tmpfs the callers already mount at ``/tmp``.
+    """
+
+    return (
+        "--user",
+        f"{os.getuid()}:{os.getgid()}",
+        "--env",
+        "HOME=/tmp/home",
+        "--env",
+        "HF_HOME=/tmp/home/hf",
+        "--env",
+        "XDG_CACHE_HOME=/tmp/home/cache",
+    )
+
+
 class RunKind(StrEnum):
     """Whether an attempt may ever be considered for commercial publication."""
 
@@ -839,7 +865,12 @@
         # The image wrapper controls the commercial stage sequence and rejects
         # an upstream setup/download path. The executor deliberately exposes no
         # caller override that could bypass the sealed image contract.
-        from scripts.wakeword_training_image import build_stage_command
+        # Import shape differs only between the extracted archive (sibling
+        # module) and repository tests (scripts package), as in the launcher.
+        try:
+            from wakeword_training_image import build_stage_command
+        except ModuleNotFoundError:
+            from scripts.wakeword_training_image import build_stage_command
 
         stage_command = build_stage_command(context.run_kind.value)
         container_name = context.container_name
@@ -859,6 +890,7 @@
             "ALL",
             "--security-opt",
             "no-new-privileges",
+            *unprivileged_container_args(),
             "--env",
             "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True",
             "--mount",

Image (requires rebuild + re-pin)

COPY --chmod=0644 deploy/wakeword-training/run-commercial-pipeline.py ./run-commercial-pipeline.py
# ...same for every COPY into /opt/jr-wakeword

Optionally move HOME / HF_HOME / XDG_CACHE_HOME into the Dockerfile as ENV and drop them from the launcher, since a rebuild is required regardless.

Rejected alternative

Keeping uid 0 and adding --cap-add DAC_OVERRIDE avoids the image rebuild and does fix the write failures - but not defect 3. Root still creates 0600 root:root files that the uid-1000 host inventory cannot hash, which is the original exit-2. --user is the only complete answer.

Verification

With the launcher patch plus an image whose /opt/jr-wakeword is world-readable, a full local experiment at default umask 022 - no chmod, no wrapper, no workaround:

status          : completed
stage           : sealed
exit_code       : 0

exports/<release>-public.tar.gz            classifier.onnx, manifest.json,
                                           evaluation.json, LICENSE,
                                           THIRD_PARTY_NOTICES.md, SHA256SUMS
exports/<release>-private-retraining.tar.gz

Zero permission or read-only errors, and every artifact owned by and readable to the invoking user. Setup independently reaches exit 0 with a complete 17 GB cache (ACAV100M, 270 RIRs, 774/774 MUSAN, Piper checkpoint).

A full-scale run (50,000 steps, 10,000 samples, medium conv_attention, Piper batch 10) is in progress on the patched copy and has cleared generation, all three augmentation rounds, and feature extraction with no errors.

Note: verification used a derived image (FROM the pinned digest + chmod -R a+rX /opt/jr-wakeword) rather than a full wheelhouse rebuild. Behaviourally equivalent for this purpose, but the real change belongs in the COPY directives so the released digest carries it.

Suggested release gate

Extract the built .tar.gz / .zip into a clean directory as a non-root user and run it from there. Every blocker above reproduces on the first attempt, and defect 5 in particular cannot reproduce from a repo checkout.

## Summary Release `0.2.1-jrich-3` cannot complete asset setup **or** start training on a correctly configured host. Four separate blocking defects, plus one silent-corruption defect that would produce a bad model once the blockers are cleared. None of this is host misconfiguration and none of it is WSL-specific - a plain Linux host with the default `umask 022` fails identically. Every finding below was reproduced, bisected, and then re-verified against a patched copy that trains end to end. **Environment used for diagnosis:** Ubuntu 24.04 on WSL2, Python 3.12.9, Docker 29.7.2, RTX 3090 (driver 591.86), NVIDIA Container Toolkit working, 855 GB free. All 28 files verified against the shipped `SHA256SUMS`. --- ## 1. Container runs as uid 0 with `--cap-drop ALL` against host-owned bind mounts - BLOCKER `scripts/wakeword_trainer.py` passes `--cap-drop ALL`. The image declares no `USER`, so the container is uid 0, but `--cap-drop ALL` removes `CAP_DAC_OVERRIDE`. Root cannot write a bind-mounted directory it does not own, and the launcher creates the cache as the invoking user at mode `0755`: ``` PermissionError: [Errno 13] Permission denied: '/data/piper' error: LiveKit asset setup failed ``` Bisected the hardening flags individually - `--read-only` and `no-new-privileges` are both harmless here; `--cap-drop ALL` is the sole cause. The same pattern applies to the training stage's `/run` mount in `scripts/wakeword_training.py`. Simulated it directly: ``` mkdir: cannot create directory '/run/output': Permission denied touch: cannot touch '/run/pipeline-time-v.txt': Permission denied ``` ## 2. `--read-only` rootfs + `HOME=/root` with no `HF_HOME` - BLOCKER The container gets `--tmpfs /tmp` but `HOME` stays `/root` on the read-only rootfs, and no HuggingFace cache variables are set. Every HF-backed dataset fails: ``` WARNING Failed to download ACAV100M features: I/O error: Read-only file system (os error 30) WARNING Failed to download validation features: ... WARNING Failed to download RIRs: ... WARNING Failed to download MUSAN noise: ... INFO Setup complete! ``` After a "successful" setup the cache held **167 MB instead of ~17 GB** - only the Piper checkpoint, which uses plain HTTP rather than `huggingface_hub`. ## 3. Root-created `0600` files break the host-side inventory - BLOCKER Even when writes succeed, the container creates `local_dir/.cache/huggingface/trees/*.json` as `root:root` mode `0600`. The host-side inventory then cannot read them: ``` error: [Errno 13] Permission denied: .../livekit-0.2.1/backgrounds/.cache/huggingface/trees/3edcf...json ``` Real exit code 2. The resulting cache also could not be removed without going through a root container. ## 4. `/opt/jr-wakeword/*` is `0600 root:root` in the image - BLOCKER for the fix Fixing 1-3 by running the container as the invoking user immediately hits: ``` python: can't open file '/opt/jr-wakeword/run-local-experiment.py': [Errno 13] Permission denied ``` Every file the Dockerfile `COPY`s is mode `0600`: ``` 600 /opt/jr-wakeword/run-local-experiment.py 600 /opt/jr-wakeword/run-commercial-pipeline.py 600 /opt/jr-wakeword/wakeword_training_image.py 600 /opt/jr-wakeword/finalize-run.py 600 /opt/jr-wakeword/stage_evidence.py ... ``` `COPY` preserves build-context modes, and nothing in the Dockerfile loosens them. This was invisible while the container ran as root. `/opt/conda` is `755`, so setup alone survives an unprivileged uid; only the training stage breaks. **This means the permission fix cannot be launcher-only - the image must be rebuilt.** ## 5. Archive-layout import bug - BLOCKER, independent of permissions `scripts/wakeword_training.py` does a bare: ```python from scripts.wakeword_training_image import build_stage_command ``` The entry point `scripts/jrich-wakeword-trainer` puts `scripts/` on `sys.path`, not the archive root, so the `scripts.` prefix is unresolvable in an extracted archive. Training dies before any container launches: ``` terminal_reason : executor_error:ModuleNotFoundError stage : pipeline elapsed : 0.048s ``` The launcher already solves this exact problem in `scripts/wakeword_trainer.py`: ```python # Import shape differs only between the extracted # archive (sibling module) and repository tests (scripts package). try: from wakeword_training import RunKind, WakewordTrainingRequest except ModuleNotFoundError: from scripts.wakeword_training import RunKind, WakewordTrainingRequest ``` There are exactly two `scripts.`-qualified imports in the archive; one is guarded, one is not. Training therefore only ever worked from a repo checkout - **the release was never exercised from its own artifact.** --- ## 6. Failed downloads reported as success - SILENT CORRUPTION, not a blocker This one survives all the fixes above and is arguably the most dangerous. `livekit/wakeword` `cli.py` downgrades *every* download failure to a `WARNING`, then prints `Setup complete!` and exits 0. The runner seals whatever landed into the immutable source lock and treats it as a valid input set. Observed: a source lock over 1008 files / 17.8 GB in which the MUSAN background corpus was **227 of 774 files**, with nothing in the provenance record indicating a problem. Compounding it, `_download_musan_noise` short-circuits on *any* non-zero count: ```python existing = list(bg_dir.glob("**/*.wav")) if existing: logger.info(f"Background noise already present: {len(existing)} files in {bg_dir}") return ``` So one transient HTTP 429 permanently pins the corpus at a partial count, and every subsequent setup reports success. Recovery requires deleting the entire `backgrounds/` directory and re-pulling all 1.1 GB. Reproduced twice (227/774, then 768/774); a wipe-and-retry loop was needed to reach 774/774. There is also no `HF_TOKEN` passthrough and no retry on 429. Unauthenticated pulls rate-limit reliably at this data volume: ``` WARNING Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. ``` **Suggested:** treat an incomplete asset set as fatal, verify counts against the expected manifest before sealing a source lock, and add token passthrough with retry/backoff. --- ## 7. Windows entry point is blocked by execution policy - end-user papercut Under the default `RemoteSigned` policy for CurrentUser, a downloaded `start-windows.ps1` carries mark-of-the-web and "Run with PowerShell" fails before anything starts: ``` File ...\start-windows.ps1 cannot be loaded. The file is not digitally signed. You cannot run this script on the current system. ``` Users can `Unblock-File`, but the README says to right-click -> Run with PowerShell, which does not work on a stock machine. Consider signing the script, shipping a `.cmd` shim, or documenting the unblock step. --- ## Proposed fix ### Launcher (this repo) Applies cleanly to a pristine `0.2.1-jrich-3` archive with `patch -p1`: ```diff --- a/scripts/wakeword_trainer.py +++ b/scripts/wakeword_trainer.py @@ -83,6 +83,8 @@ ) -> tuple[str, ...]: """Build the explicit networked cache-population command; training stays offline.""" + from wakeword_training import unprivileged_container_args + cache_dir.mkdir(parents=True, exist_ok=True) return ( "docker", @@ -96,6 +98,7 @@ "ALL", "--security-opt", "no-new-privileges", + *unprivileged_container_args(), "--mount", f"type=bind,src={cache_dir.resolve()},dst=/data", "--mount", @@ -352,6 +355,8 @@ def _preflight(plan: dict[str, Any]) -> None: + from wakeword_training import unprivileged_container_args + runtime = plan["runtime"] for executable in ("docker", "nvidia-smi"): if shutil.which(executable) is None: @@ -384,6 +389,9 @@ "ALL", "--security-opt", "no-new-privileges", + # Preflight under the same uid the real stages use, so this check + # cannot pass in a configuration training will not run in. + *unprivileged_container_args(), runtime["image_ref"], "python", "-c", --- a/scripts/wakeword_training.py +++ b/scripts/wakeword_training.py @@ -73,6 +73,32 @@ ) +def unprivileged_container_args() -> tuple[str, ...]: + """Run the container as the invoking user with caches off the read-only rootfs. + + The image declares no ``USER``, so without this the container is uid 0 while + ``--cap-drop ALL`` removes ``CAP_DAC_OVERRIDE``. Root then cannot write a + bind-mounted host directory it does not own, and any file it does create is + unreadable to the host user that has to inventory it afterwards. Matching the + host uid/gid keeps both directions working under the same dropped capabilities. + + ``HOME`` defaults to ``/root`` on the read-only rootfs, so HuggingFace and + friends fail with ``Read-only file system``. Point the HOME-derived cache + paths at the writable tmpfs the callers already mount at ``/tmp``. + """ + + return ( + "--user", + f"{os.getuid()}:{os.getgid()}", + "--env", + "HOME=/tmp/home", + "--env", + "HF_HOME=/tmp/home/hf", + "--env", + "XDG_CACHE_HOME=/tmp/home/cache", + ) + + class RunKind(StrEnum): """Whether an attempt may ever be considered for commercial publication.""" @@ -839,7 +865,12 @@ # The image wrapper controls the commercial stage sequence and rejects # an upstream setup/download path. The executor deliberately exposes no # caller override that could bypass the sealed image contract. - from scripts.wakeword_training_image import build_stage_command + # Import shape differs only between the extracted archive (sibling + # module) and repository tests (scripts package), as in the launcher. + try: + from wakeword_training_image import build_stage_command + except ModuleNotFoundError: + from scripts.wakeword_training_image import build_stage_command stage_command = build_stage_command(context.run_kind.value) container_name = context.container_name @@ -859,6 +890,7 @@ "ALL", "--security-opt", "no-new-privileges", + *unprivileged_container_args(), "--env", "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True", "--mount", ``` ### Image (requires rebuild + re-pin) ```dockerfile COPY --chmod=0644 deploy/wakeword-training/run-commercial-pipeline.py ./run-commercial-pipeline.py # ...same for every COPY into /opt/jr-wakeword ``` Optionally move `HOME` / `HF_HOME` / `XDG_CACHE_HOME` into the Dockerfile as `ENV` and drop them from the launcher, since a rebuild is required regardless. ### Rejected alternative Keeping uid 0 and adding `--cap-add DAC_OVERRIDE` avoids the image rebuild and does fix the write failures - but not defect 3. Root still creates `0600 root:root` files that the uid-1000 host inventory cannot hash, which is the original exit-2. `--user` is the only complete answer. --- ## Verification With the launcher patch plus an image whose `/opt/jr-wakeword` is world-readable, a full local experiment at default umask 022 - no `chmod`, no wrapper, no workaround: ``` status : completed stage : sealed exit_code : 0 exports/<release>-public.tar.gz classifier.onnx, manifest.json, evaluation.json, LICENSE, THIRD_PARTY_NOTICES.md, SHA256SUMS exports/<release>-private-retraining.tar.gz ``` Zero permission or read-only errors, and every artifact owned by and readable to the invoking user. Setup independently reaches exit 0 with a complete 17 GB cache (ACAV100M, 270 RIRs, 774/774 MUSAN, Piper checkpoint). A full-scale run (50,000 steps, 10,000 samples, medium `conv_attention`, Piper batch 10) is in progress on the patched copy and has cleared generation, all three augmentation rounds, and feature extraction with no errors. > Note: verification used a derived image (`FROM` the pinned digest + `chmod -R a+rX /opt/jr-wakeword`) rather than a full wheelhouse rebuild. Behaviourally equivalent for this purpose, but the real change belongs in the `COPY` directives so the released digest carries it. ## Suggested release gate Extract the built `.tar.gz` / `.zip` into a clean directory as a non-root user and run it from there. Every blocker above reproduces on the first attempt, and defect 5 in particular cannot reproduce from a repo checkout.
westtexasfish commented 2026-08-14 12:51:04 -05:00
Author
Owner
Copy Link
Copy Source

Reproduced the two deterministic release failures from v0.2.1-jrich-3: the launcher emits root-container commands without a writable HOME/Hugging Face cache, and the extracted archive fails to resolve scripts.wakeword_training_image when launched outside its package root. A source fix is in progress on ai/jrich-gateway branch fix/wakeword-trainer-wsl-release; it will include non-root bind-mount execution, cache validation/retry, archive-layout coverage, a Windows launcher fallback, and clean-extraction release tests.

Reproduced the two deterministic release failures from v0.2.1-jrich-3: the launcher emits root-container commands without a writable HOME/Hugging Face cache, and the extracted archive fails to resolve `scripts.wakeword_training_image` when launched outside its package root. A source fix is in progress on `ai/jrich-gateway` branch `fix/wakeword-trainer-wsl-release`; it will include non-root bind-mount execution, cache validation/retry, archive-layout coverage, a Windows launcher fallback, and clean-extraction release tests.
westtexasfish commented 2026-08-14 13:34:04 -05:00
Author
Owner
Copy Link
Copy Source

The corrected artifacts are now published as v0.2.1-jrich-4, backed by public image digest sha256:940abc406888079c33778b0c8c34ff594b97998a3ab1c7a09faa9acf4e3d6781. The source fix is ai/jrich-gateway PR #187.

Verified gates include UID 12345 execution with the production Docker restrictions, legacy root-owned cache repair, the complete reference cache (4 singleton hashes / 270 RIR / 774 MUSAN), exact public download hashes, tar and ZIP extraction with deterministic modes, archive-only imports from another working directory, and a ZIP path containing spaces. I am leaving the issue open until PR CI/review is complete.

The corrected artifacts are now published as [v0.2.1-jrich-4](https://git.jimandkrista.com/jr-public/jrich-wakeword-trainer/releases/tag/v0.2.1-jrich-4), backed by public image digest `sha256:940abc406888079c33778b0c8c34ff594b97998a3ab1c7a09faa9acf4e3d6781`. The source fix is [ai/jrich-gateway PR #187](https://git.jimandkrista.com/ai/jrich-gateway/pulls/187). Verified gates include UID 12345 execution with the production Docker restrictions, legacy root-owned cache repair, the complete reference cache (4 singleton hashes / 270 RIR / 774 MUSAN), exact public download hashes, tar and ZIP extraction with deterministic modes, archive-only imports from another working directory, and a ZIP path containing spaces. I am leaving the issue open until PR CI/review is complete.
westtexasfish commented 2026-08-14 14:08:58 -05:00
Author
Owner
Copy Link
Copy Source

WSL portability fixes are implemented and verified in jrich-gateway PR #187.

Published replacement release: v0.2.1-jrich-4

The release now:

  • runs trainer containers as the invoking WSL/Linux UID:GID so bind-mounted files remain writable and host-owned;
  • provides writable HOME, Hugging Face, and XDG cache paths under the mounted cache;
  • validates pinned, complete training assets and fails closed on partial/failed downloads;
  • repairs legacy root-owned cache/work files with a narrowly scoped CAP_CHOWN container;
  • has portable archive imports and normalized archive permissions;
  • includes a Windows .cmd launcher that handles PowerShell execution-policy friction;
  • is available as both .tar.gz and .zip.

Verification included the exact public archives, a non-root read-only/cap-drop container run, legacy ownership repair, execution from a different working directory and a path containing spaces, asset revision/count/hash validation, and the full PR CI matrix. Fresh CI is green: SQLite, Postgres (1,320 tests), web build, gateway image build, and Playwright E2E. The issue should remain open until PR #187 is human-merged and the gateway UI points at this release.

WSL portability fixes are implemented and verified in [jrich-gateway PR #187](https://git.jimandkrista.com/ai/jrich-gateway/pulls/187). Published replacement release: [v0.2.1-jrich-4](https://git.jimandkrista.com/jr-public/jrich-wakeword-trainer/releases/tag/v0.2.1-jrich-4) The release now: - runs trainer containers as the invoking WSL/Linux UID:GID so bind-mounted files remain writable and host-owned; - provides writable HOME, Hugging Face, and XDG cache paths under the mounted cache; - validates pinned, complete training assets and fails closed on partial/failed downloads; - repairs legacy root-owned cache/work files with a narrowly scoped CAP_CHOWN container; - has portable archive imports and normalized archive permissions; - includes a Windows `.cmd` launcher that handles PowerShell execution-policy friction; - is available as both `.tar.gz` and `.zip`. Verification included the exact public archives, a non-root read-only/cap-drop container run, legacy ownership repair, execution from a different working directory and a path containing spaces, asset revision/count/hash validation, and the full PR CI matrix. Fresh CI is green: SQLite, Postgres (1,320 tests), web build, gateway image build, and Playwright E2E. The issue should remain open until PR #187 is human-merged and the gateway UI points at this release.
westtexasfish commented 2026-08-14 15:30:35 -05:00
Author
Owner
Copy Link
Copy Source

Resolved and deployed. jrich-gateway PR #187 was merged as f2d244b; the complete post-merge CI matrix passed, live deployment completed at migration head 0019, gateway and web both returned HTTP 200, and rollback tag live-20260814-3 was created. The live UI now references the corrected v0.2.1-jrich-4 release.

Resolved and deployed. [jrich-gateway PR #187](https://git.jimandkrista.com/ai/jrich-gateway/pulls/187) was merged as `f2d244b`; the complete post-merge CI matrix passed, live deployment completed at migration head `0019`, gateway and web both returned HTTP 200, and rollback tag `live-20260814-3` was created. The live UI now references the corrected [v0.2.1-jrich-4 release](https://git.jimandkrista.com/jr-public/jrich-wakeword-trainer/releases/tag/v0.2.1-jrich-4).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
claude-agent (Claude-Agent) westtexasfish (westtexasfish)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: jr-public/jrich-wakeword-trainer#1