Migrate to new Zig compiler by lukewilliamboswell · Pull Request #413 · roc-lang/basic-cli

lukewilliamboswell · 2025-12-30T02:06:50Z

No description provided.

Removed Path.type! function which was causing crash: "increfDataPtrC: ORIGINAL ptr=0x1 is not 8-byte aligned". This appears to be an ABI issue with opaque types in RocTry with the new compiler. The function used an opaque type PathType := [IsDir, IsFile, IsSymLink] which doesn't work correctly in Try results. The three individual functions (is_file!, is_dir!, is_sym_link!) all work perfectly and cover the same use cases. All tests now pass. Path.type! can be investigated as a separate follow-up issue once the opaque type ABI is better understood. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

- Restructure Cmd to use opaque type with methods (Cmd := { record }.{ methods }) - Update doc comments to use -> operator syntax instead of deprecated |> - Delete orphaned examples (dir-test, env-test, file-test, print-test) - Delete orphaned expect scripts for removed examples - Delete old tests/ directory with unmigrated tests - Update all_tests.sh to remove references to deleted examples - Add test files demonstrating -> operator syntax works Note: Static dispatch with . syntax crashes compiler (checkDeferredStaticDispatchConstraints) so we use -> operator syntax instead, which works correctly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

After compiler fixes for static dispatch, we can now use the idiomatic . syntax for all methods including effects: - Update all doc comments to show static dispatch syntax - Effect methods now work: cmd.exec_exit_code!() - Builder methods continue to work: Cmd.new("ls").args([]) - Full fluent API: Cmd.new("echo").args(["Hi"]).exec_cmd!() Previous syntax required: - . for regular methods - Qualified calls for effects (Cmd.exec_exit_code!(cmd)) Now all methods use consistent . syntax throughout. Tests verify: - Simple method chaining with effects - Multiline builder patterns - Output capture via static dispatch - Complete fluent chains 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Use argc/argv directly instead of std::env::args() because when built as a static library, the Rust runtime isn't properly initialized and std::env::args() returns an empty list. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

c_char is u8 on ARM64 Linux but i8 on x86, so use the portable c_char type from std::ffi for argv handling. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The `git diff | head -50` command can exit with 141 (SIGPIPE) on Linux when head closes the pipe before git finishes writing. Add `|| true` to ignore this expected condition. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Limit GITHUB_TOKEN permissions to contents:read by default, as recommended by GitHub's security scanner. The create-release job overrides this with contents:write as needed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Anton-4 · 2026-01-08T18:41:18Z

-    pub stdout_bytes: roc_std::RocList<u8>,
+pub struct CommandOutputSuccess {
+    pub stderr_utf8_lossy: RocStr,  // offset 0 (24 bytes)
+    pub stdout_utf8: RocStr,        // offset 24 (24 bytes)


@lukewilliamboswell in several places in the PR, Str is used where there was a byte list before. What's the reasoning behind that?

We haven't finished migrating things across yet. The RocStr was working across the host boundary and simpler place to start with. I can't remember if I tried the RocList first and it was broken, but I've just been trying to change one thing at a time. Now we have a functional CI it will be easier to make changes and know if we break anything.

GitHub Actions bash uses set -e by default, which caused the script to exit immediately when the command substitution failed, before the output could be printed. This made debugging impossible. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

roc bundle does not accept absolute paths for input files. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

roc bundle doesn't allow path traversal (..) so we copy the file into the platform directory and clean it up on exit. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add Tty/Locale modules and example tests * Add comments about raw mode * set specific version --------- Co-authored-by: Anton-4 <17049058+Anton-4@users.noreply.github.com>

Anton-4 · 2026-02-11T18:21:53Z

Status update: I am still working on this regularly but I have not made much progress yet.

Anton-4 · 2026-02-20T19:06:52Z

See #423 for updates

WIP edits to migrate-zig-compiler branch

Brings SQLite back online under the new flat-platform architecture (it had been dropped during the zig-compiler migration and only survived as disabled `.todoroc` examples). - platform/InternalSqlite.roc, platform/Sqlite.roc: host-ABI types plus the full decoder/combinator/lifecycle API (prepare!/bind!/columns!/column_value!/ step!/reset! + execute!/query!/query_many!/prepared variants, leaf + nullable decoders, decode_record/map_value/map_value_result, ErrCode/errcode_to_str). The opaque `Stmt :: Box(U64)` handle stores the raw statement pointer. - src/lib.rs: SQLite host fns over libsqlite3-sys (bundled) using the box-payload handle helpers, a per-path connection cache, and value/error marshalling. - Cargo.toml: add libsqlite3-sys = "=0.33.0" (bundled). - examples/sqlite-basic.roc, sqlite-everything.roc (+ todos.db / todos2.db fixtures): migrated from the old syntax; both run green. - ci: run sqlite-basic in the expect suite. Note: the regenerated src/roc_platform_abi.rs requires the upstream RustGlue.roc box-header fix (decref_box_with now takes an explicit payload_contains_refcounted flag instead of inferring it from payload_decref.is_some(); the old inference freed Box(U64) host handles from the wrong base and segfaulted). basic-cli must be regenerated against a roc that includes that fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Bring TCP back online under the new flat platform architecture, mirroring the SQLite port: - platform/Tcp.roc: opaque `Stream :: Box(U64)` handle, 5 `host_*!` FFI declarations, and ergonomic wrappers (connect!/read_up_to!/read_exactly!/ read_until!/read_line!/write!/write_utf8!) plus ConnectErr/StreamErr unions and parse_*/*_to_str helpers. - src/lib.rs: 5 hosted_tcp_* host fns over a boxed BufReader<TcpStream>, using the same box helpers proven for SQLite (false for payload_contains_refcounted), with errors marshalled as Str. - platform/main.roc: expose Tcp, import it, and append the hosted_tcp_* entries at the end of the hosted block to avoid renumbering glue types. - src/roc_platform_abi.rs: regenerated (no existing aliases shifted). - examples/tcp-client.roc: migrated from .todoroc to new-compiler syntax. - ci/expect_scripts/tcp-client.exp + tcp_echo_server.py: self-contained expect test using a python3 echo server (no ncat dependency); wired into EXPECT_EXAMPLES in ci/all_tests.sh. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Bring HTTP online under the flat platform, validating that the rustls/ring TLS stack cross-compiles and runs under musl (the reason for this branch). - platform/InternalHttp.roc: HTTP data types + host-boundary record shapes (types only, mirroring InternalSqlite). - platform/Http.roc: host_send_request! plus default_request/header/send!/ get_utf8! and the private Method<->U64 / timeout converters. - src/lib.rs: hosted_http_send_request over a thread-local current-thread tokio runtime + hyper + hyper-rustls (with_native_roots, ring). Transport failures encoded as status+body sentinels decoded by send!. Also re-point the glue aliases that renumbered when the Http/InternalHttp modules were added (stdin err 103/108->112/117, SqliteError/Bindings 76/84->85/93). - Cargo.toml: pinned hyper/hyper-util/hyper-rustls/http-body-util/tokio/bytes. - Cargo.lock: downgrade transitive deps that require edition2024 (unbuildable on the pinned 1.82 toolchain) back to main's resolved versions. - examples/http-client.roc: HTTPS GET demo. Validated: full dep stack compiles for x86_64-unknown-linux-musl via the zig cc wrapper, and the musl-linked example completes a real HTTPS handshake to example.com (and plain HTTP to a local server). SQLite/TCP still build+run. Not yet done: get!/json (Decoding abilities), ci/rust_http_server, expect test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

- platform/Http.roc: use the new spread record-update syntax (`{ ..default_request, uri }`) in get_utf8!. - examples/http-client.roc: end-to-end demo exercising get_utf8! (plain text + JSON) and send! (custom header + Response record/status) against the local test server. - ci/rust_http_server/: restored the hyper-based test server (serves /utf8test -> "Hello utf8" and / -> {"foo":"Hello Json!"} on :9000). - ci/expect_scripts/http-client.exp: starts the server and verifies the example's output; wired into EXPECT_EXAMPLES, with all_tests.sh building the server up front. Verified e2e: example prints the expected three lines and the expect test passes; all examples check+build and the generated glue is up to date. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

lukewilliamboswell · 2026-06-26T08:35:50Z

Next target: JSON parsing

With SQLite, TCP, and the core HTTP client (Http.send! / Http.get_utf8!) now ported and green, the next gap is JSON.

The old Http.get! and the examples/http.todoroc demo both relied on Decode.from_bytes with the Decoding / DecoderFormatting abilities plus the external roc-json package. Neither has been validated against the new Zig compiler yet:

Abilities (where body implements Decoding, fmt implements DecoderFormatting) are untried on the new compiler.
The roc-json package likely doesn't build on the new compiler in its current form.

So the next piece of work is to either port/port-validate a JSON parsing path (abilities + roc-json, or an alternative), then restore Http.get! and migrate examples/http.todoroc. The core HTTP transport (including HTTPS via rustls/ring, validated under musl) is already in place and doesn't depend on this.

lukewilliamboswell added 10 commits December 30, 2025 12:42

initial migration - support just Sdout and native target

6ba1155

migrate Env and File effects, examples and tests

b1f3ce5

migrate test script

5779c84

migrate command line example and test

c2b3e05

add linux binaries libc and libunwind

519d2af

add linux CRT files

4774794

migrate print-test

08e14d4

migrate Dir functionality

e914e0a

migrate Path

41f75f5

update CI

3e6246a

github-advanced-security AI found potential problems Dec 30, 2025

View reviewed changes

Comment thread .github/workflows/ci.yml Fixed

lukewilliamboswell added 4 commits December 30, 2025 17:23

improve error handling, use open tag union

4f62b3b

implement Random, Sleep, Utc modules

1fcef27

migrate Cmd

953ed5f

fix some things

8c951de

github-advanced-security AI found potential problems Jan 5, 2026

View reviewed changes

Comment thread .github/workflows/release.yml Fixed

Comment thread .github/workflows/release.yml Fixed

lukewilliamboswell and others added 10 commits January 5, 2026 15:24

Delete test-cmd-method-chaining

41f87a4

Make Cmd an opaque nominal type

6db4a74

remove nix for now

c1d5baf

Anton-4 mentioned this pull request Jan 7, 2026

Fix stdin performance issue #416

Open

Anton-4 added 3 commits January 7, 2026 19:41

small CI improvements

28a376e

same setup-zig version everywhere

188e1b5

use flaky-retry action

c0d20d9

Anton-4 mentioned this pull request Jan 8, 2026

licenses file needs to be included in release archive #417

Open

Cargo.toml improvements

8fd545b

Anton-4 reviewed Jan 8, 2026

View reviewed changes

Anton-4 and others added 7 commits January 12, 2026 18:11

gitignore binary files without extension

070b8b3

bundle include THIRD_PARTY_LICENSES.md #417

8c1135c

use commit for latest nightly

43cdaf5

use nightly roc

2b4c74f

fix: use relative path for THIRD_PARTY_LICENSES.md in bundle

634720e

roc bundle does not accept absolute paths for input files. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

fix: copy THIRD_PARTY_LICENSES.md into platform dir for bundling

03e94ad

roc bundle doesn't allow path traversal (..) so we copy the file into the platform directory and clean it up on exit. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Anton-4 changed the title ~~Migrate zig compiler~~ WIP Migrate zig compiler Jan 13, 2026

Anton-4 reviewed Jan 13, 2026

View reviewed changes

Comment thread platform/Cmd.roc Outdated

Add Tty and Locale modules (#418)

a2e0191

* Add Tty/Locale modules and example tests * Add comments about raw mode * set specific version --------- Co-authored-by: Anton-4 <17049058+Anton-4@users.noreply.github.com>

Anton-4 self-assigned this Jan 28, 2026

Anton-4 reviewed Jan 28, 2026

View reviewed changes

Comment thread platform/Host.roc Outdated

lukewilliamboswell added 4 commits June 25, 2026 11:00

Upgrade Roc glue and CI harness

6ff2359

Port API updates to migrate zig compiler

2305a8e

Merge pull request #423 from roc-lang/migrate-zig-compiler-edits

e25412e

WIP edits to migrate-zig-compiler branch

Clean up migrate zig compiler branch

80ee298

lukewilliamboswell changed the title ~~WIP Migrate zig compiler~~ Migrate to new Zig compiler Jun 25, 2026

lukewilliamboswell and others added 6 commits June 25, 2026 15:21

Prepare migrated platform for release CI

b5aa0d9

Pin Roc glue alias generator

0b8f3e1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Migrate to new Zig compiler#413

Migrate to new Zig compiler#413
lukewilliamboswell wants to merge 46 commits into
mainfrom
migrate-zig-compiler

lukewilliamboswell commented Dec 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Anton-4 Jan 8, 2026

Uh oh!

lukewilliamboswell Jan 9, 2026

Uh oh!

Uh oh!

Uh oh!

Anton-4 commented Feb 11, 2026

Uh oh!

Anton-4 commented Feb 20, 2026

Uh oh!

lukewilliamboswell commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

lukewilliamboswell commented Dec 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Anton-4 Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

lukewilliamboswell Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Anton-4 commented Feb 11, 2026

Uh oh!

Anton-4 commented Feb 20, 2026

Uh oh!

lukewilliamboswell commented Jun 26, 2026

Next target: JSON parsing

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants