Commit Graph
100 Commits
Author SHA1 Message Date
Jonathan A. Sternberg 0a0fc905a6 feat: add persistent storage options to k8s driver
The k8s driver now supports the `persistent-volume-claim.requests.storage`
option. Setting this option changes the deployment into a statefulset
and creates a persistent volume claim where the buildkit data is stored.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-04-01 15:44:51 -05:00
Jonathan A. Sternberg a8d359a9ca dap: make dap generally available
Removes the experimental flags and bits for dap and deletes some dead
code that somehow made its way this far without anyone noticing.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-23 10:43:27 -05:00
Jonathan A. SternbergandGitHub dc5f9862a5 Merge pull request #3735 from jsternberg/dap-exit-code
dap: pass exit code through exited event
2026-03-23 10:42:57 -05:00
Jonathan A. Sternberg 180cfd9e41 dap: pass exit code through exited event
Pass the exit code through the exited event back to the client and
ensure that the printed text is printed completely.

Previously, the exited event just had a big todo and the printer would
sometimes fail to send messages to the connected client. This moves the
printer wait to before the debug adapter is closed to ensure that all
messages get sent through the connection to the editor. While there, I
also plumbed in the exit code to exited. It's not necessarily the real
exit code but it will produce a zero on build success and a non-zero
code on build failure so that should be good enough.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-20 14:02:21 -05:00
Jonathan A. Sternberg c79061526f dap: defer inputs for a step to prevent overeager evaluation
When the debug thread was updated to always solve inputs from the
operation that it was tied to it became a bit overeager to evaluate
them. The intention of the steps is to have a single direct parent and
then potentially multiple "function calls" that can be evaluated with
step into and step out to leave.

With the change, that logic stayed in, but the inputs were always being
evaluated before they were stepped into or over. Now, when we construct
the steps, we also attach a list of inputs that we should defer
evaluation on to ensure we don't execute inputs that haven't been
executed yet.

It will then wrap the reference with a version that causes `Evaluate` to
do nothing. This prevents the overeager evaluation but allows the
reference to be evaluated if we need to read the filesystem.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-20 09:26:32 -05:00
Jonathan A. Sternberg b33ef2c9d4 dap: fix the check to determine whether exec will succeed
This refines the check for determining whether exec will succeed to work
when an error occurs. This check previously relied on the `Ref` being
populated in the result context but this would only happen if we were
paused from a breakpoint or by stepping. An error would not fill in this
field.

The check is now refined to use the new gateway filesystem exec API so
we can create the container and then check even if we don't have a
returned gateway reference. The logic to determine which mount to check
has also been moved.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-17 10:31:11 -05:00
Jonathan A. Sternberg 8e29ab9493 ci: remove test-bsd-unit workflow
This workflow doesn't provide much value. It mostly fails on temporary
errors regarding vagrant and doesn't provide a very useful signal that
the build is broken.

We perform cross-compilation and ensure `buildx` still compiles on
different bsd variants. That should be enough to make sure we don't
inadvertently break something.

Closes #3711.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-12 10:36:36 -05:00
Jonathan A. Sternberg 724afbb867 dap: fix skipped breakpoint when the breakpoint and the entrypoint were the same
We erroneously skipped a breakpoint when that breakpoint was the same as
the entrypoint and we did not use stop on entry. This is because we only
started evaluating breakpoints after the first step on the entrypoint
instead of at the entrypoint.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-11 10:52:50 -05:00
Jonathan A. Sternberg 3e4bd229ab dap: skip the load build context step when it doesn't have an associated source line
Skip the load build context step when it doesn't have an associated
source line. This caused an extra branch to be created in an otherwise
pretty straightforward dockerfile where stepping in on a copy
instruction that used the context would stay on the same line because it
"stepped into" the context loading rather than being treated the same as
step next.

This resulted in some bad and confusing ergonomics with the cursor
position that were a bit confusing and unexpected.

There might be more areas to try and prune but the most common one, a
single branch instruction that doesn't have a location, now gets skipped
which is the exact thing that was generated for loading the context.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-10 12:35:13 -05:00
Jonathan A. Sternberg ca9df87014 dap: fix the race condition in the dap unit tests
The context used for serving the dap server was being canceled too early
because it used defer which would initiate at the end of the function
while every other cleanup function used `t.Cleanup` which executes in
its own goroutine.

One possible solution was to move the cancel to the cleanup, but the
context being passed to serve and start doesn't make sense because if it
ever does get canceled, it'll likely cause a similar race condition with
`Stop`.

This removes the context from the methods that were causing this issue
in favor of just relying on the caller calling `Stop` when they are done
with the adapter and server.

This seems to have only affected tests and I don't believe it affected
the actual dap command.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-10 12:03:35 -05:00
Jonathan A. Sternberg 4f3de79c13 dap: properly map source paths to client side paths
Properly map the source paths from the metadata in the solve to the
client side paths. The source path returns is relative to the context
that gets uploaded which is usually a subdirectory. The original code
noticed this when mapping the paths but made the invalid assumption that
the dockerfile would always be in the context path so it combined the
dockerfile name with the context path.

It is possible for the dockerfile to be in a subdirectory of the
context. In which case, we computed the paths incorrectly.

This modifies DAP to instead use the `DockerfileMappingDst` and
`DockerfileMappingSrc` which are special included variables to the
inputs that get filled in during the build for the purpose of mapping
the source path to the client side path.

Tests have also been added for this functionality to ensure it doesn't
break again. This should work with both absolute and relative paths
although absolute paths should probably be preferred for usage just
because they're less likely to result in weird things happening.

The sources are also normalized to always convert the source filenames
to absolute paths and DAP itself will accept relative paths but will
only ever communicate in absolute paths. When you set a breakpoint, it
will convert it to an absolute path and reference it in that way rather
than a relative path.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-10 09:17:42 -05:00
Jonathan A. Sternberg b521a083ba dap: detect breakpoints for files when the case differs
Case insensitive filesystems can cause breakpoints to not be seen or
verified. This is particularly true on Windows where the drive letter
can also participate in the filepath.

Change the detection logic for a breakpoint to be case insensitive. At
the same time, report the name of the source as part of the breakpoint
so that the editor can be told which casing we're expecting to be used.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-03-06 14:01:32 -06:00
Jonathan A. Sternberg 1da36d13e8 dap: use container fs requests to get a more accurate state for the file system
The container filesystem request API that has been added to buildkit
allows a container created through the `NewContainer` API to also access
the filesystems. This is useful when an error occurs because it allows
us to grab the mutable state of the mounts used during the actual build
rather than the input version copies which don't contain any files that
were added as part of the failed command.

This gives us a more accurate view of the filesystem that was previously
only accessible through using `exec` and `ls`/`cat` commands that may
not always exist.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-02-05 09:40:55 -06:00
Jonathan A. Sternberg 3e0e9333c2 commands: adds metrics associated with the debugger
Add metrics associated with the debugger that are reported through the
metrics writer. This adds a few attributes that are only added when a
debugger is used with either the `debug` command or `dap` command.

At the moment, these metrics show up the exact same as a build and we
can't identify if something is using `dap` or `debug` since they use the
same code path.

This also adds a new available metric that can be utilized by plugins to
report additional information. The metrics will check if an environment
variable `BUILDX_DAP_USER_AGENT` is sent and that will get included in
the metrics if they are enabled.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-01-28 14:05:11 -06:00
Jonathan A. Sternberg 30406735a5 dap: improve file explorer source names
Improves the naming for file explorer names when the input relates
directly to a source. This is most common when the input is the context
(which is just usually a simple source like `local://context`).

This should help in most circumstances in determining which input is
which.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-01-28 10:04:37 -06:00
Jonathan A. Sternberg 2e76c05592 build: handle at symbols in an oci-layout path
The buildx command line will now handle `@` symbols in the
`oci-layout://` path when used with named contexts. Instead of
assuming the `@` symbol is part of the reference digest, it will first
check that it is a valid reference digest. Otherwise, it will assume
it's part of the file path.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-01-12 11:19:51 -06:00
Jonathan A. Sternberg 8ad75dc485 dap: fill in breakpoint reason for being unverified
When a breakpoint fails to be verified, it will switch the reason to
"failed". It starts off the reason as "pending".

The `reason` field for a breakpoint was added some time after the last
release of `go-dap` which has only been updated once in the last year so
this uses the `main` branch version which contains the field.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-01-08 15:21:54 -06:00
Jonathan A. Sternberg 19eadba5fc dap: fix the run mount input names
The mounts for exec operations was misconstruing most input names as the
root mount because it was using the wrong input index to match with the
exec mounts.

The correct input index is now being used so bind mounts and other types
of mounts should now show the correct mount destination rather than only
showing the root mount.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2026-01-06 12:29:51 -06:00
Jonathan A. Sternberg fdfba3014d tests: add integration tests for dap build
This adds integration tests for the `dap build` command to test various
behavior associated with the command. We start the build and the
integration test acts as a dap client to send requests and check that
the output is what we expect.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-12-17 10:56:30 -06:00
Jonathan A. Sternberg 17952c617d chore: remove unneeded excludes
The upstream PR that used the pseudo-versions was merged so this no
longer causes an upgrade of those dependencies to pseudo-versions so the
exclude is no longer needed.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-12-04 10:46:27 -06:00
Jonathan A. Sternberg b1cc133cd9 dockerfile: buildkit v0.26.0
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-11-12 16:08:34 -06:00
Jonathan A. SternbergandGitHub a2cf0edfb4 Merge pull request #3516 from jsternberg/vendor-buildkit
vendor: github.com/moby/buildkit v0.26.0
2025-11-12 13:03:44 -06:00
Jonathan A. Sternberg fd8b77f36b vendor: github.com/moby/buildkit v0.26.0
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-11-12 12:38:10 -06:00
Jonathan A. SternbergandGitHub c1fbb49e9b Merge pull request #3511 from tonistiigi/bake-nil-panic
build: avoid panic on linking to nil target
2025-11-11 15:36:05 -06:00
Jonathan A. SternbergandGitHub 8e8054f1d9 Merge pull request #3510 from tonistiigi/fix-mediatype-concurrency
imagetools: fix concurrent map write from containerd issue
2025-11-11 15:35:29 -06:00
Jonathan A. SternbergandGitHub c9b2da6ba4 Merge pull request #3513 from jsternberg/vendor-buildkit
vendor: update buildkit v0.26.0-rc2
2025-11-11 15:31:33 -06:00
Jonathan A. Sternberg 1651da02d9 vendor: update buildkit v0.26.0-rc2
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-11-11 14:14:36 -06:00
Jonathan A. Sternberg 9099a030f4 dap: attempt to find multiple digests when a failure happens
The solve error that gets returned sometimes has a platform when the LLB
returned by `ToState` doesn't. In order to ensure we find the failed
digest, we now search for both the digest of what was returned and also
clear out the platform and search for the digest without that in case
that one matches instead.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-10-29 16:08:46 -05:00
Jonathan A. Sternberg 5b8b65a59c dap: next and out now respect breakpoints
If a breakpoint occurs before the step pointed to by next or out, dap
will now stop there instead of the desired location.

This also updates the loop to always set the breakpoints rather than
only when continue is chosen.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-10-29 15:20:15 -05:00
Jonathan A. SternbergandGitHub 166ab97882 Merge pull request #3481 from rcjsuen/dap-set-breakpoints-null-fix
dap: stop sending null to conform to the specification
2025-10-24 13:43:45 -05:00
Jonathan A. Sternberg 4345a8e50b dap: refactor how steps are evaluated in the program
This change modifies how steps are evaluated in the program. Previously,
the execution relied on looking for the digest we were going to stop at
and evaluating the step right before that digest. This could result in
some gaps where it erroneously didn't execute inputs for the current
step as it skipped those digests.

Now, the evaluation reads the inputs and executed those directly rather
than relying on the evaluation of previous steps in the sequence. This
should make the evaluation of inputs more accurate and allow us to have
better breakpoints on things like the copy operation.

Due to the change, it's also easier for us to include the different
inputs in the file explorer.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-10-24 11:13:25 -05:00
Jonathan A. SternbergandGitHub abf6ab4a37 Merge pull request #3471 from jsternberg/dap-attach-standalone
dap: support run in terminal request when buildx is run in standalone mode
2025-10-17 13:07:58 -05:00
Jonathan A. Sternberg 6ac8458692 dap: support run in terminal request when buildx is run in standalone mode
The run in terminal request sent by exec would not work if `buildx` had
been invoked directly instead of through `docker`. If we don't find the
reexec environment variable, we use `os.Args[0]` to launch buildx
directly when invoking attach.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-10-15 12:03:49 -05:00
Jonathan A. SternbergandGitHub beaebcbf39 Merge pull request #3438 from thaJeztah/bump_engine
vendor: github.com/docker/docker, docker/cli v28.5.1, buildkit v0.25.1
2025-10-09 09:54:43 -05:00
Jonathan A. Sternberg ec01355b03 docs: clarify cli help for the --tag option
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-10-06 12:46:59 -05:00
Jonathan A. Sternberg b47488aea2 ci: update buildkit ref for ci
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-10-01 09:57:59 -05:00
Jonathan A. SternbergandGitHub e2534c77e3 Merge pull request #3442 from jsternberg/buildkit-vendor
vendor: update buildkit to v0.25.0
2025-09-30 14:22:56 -05:00
Jonathan A. Sternberg 3d9cc5128a vendor: update buildkit to v0.25.0
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-09-30 14:09:19 -05:00
Jonathan A. Sternberg 92ea5e64b4 dap: file explorer uses the wrong path when looking up mounts
It accidentally retrieved the relative path to the mount directory
instead of the full path which resulted in always returning the root
directory.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-09-24 09:28:38 -05:00
Jonathan A. Sternberg 39feed47ea docs: add plugin guideline requirements for dap plugins
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-09-22 12:13:15 -05:00
Jonathan A. Sternberg 17c2e1dc35 dap: collect references for inputs to ensure the file explorer works on an error
The debug adapter will now look at the error and collect the input
mounts from the exec operation and translate them to a gateway reference
to be able to access the filesystem in the file explorer.

When we are attempting to find the real operation that failed and
succeed, we now re-solve the dependencies (which should already have
been solved correctly to begin with) so that we can have the proper
reference and can access the filesystem contents.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-09-17 15:58:28 -05:00
Jonathan A. SternbergandGitHub f55ed6015b Merge pull request #3412 from crazy-max/dap-fix-data-path
dap: use slash separator when reading ref files
2025-09-08 09:42:07 -05:00
Jonathan A. Sternberg bf40c24e00 dap: improve location resolution for duplicate digests
This improves location resolution for duplicate digests by making some
assumptions about the file structure to determine a "best match" when
there are multiple possible locations.

It uses the next operation (that this digest is the input for) to
determine a location. If the location of the current operation is before
the next operation, this location is preferred. If there are multiple
locations that happen before the next operation, the one closest to the
next operation (aka later in the file) is used instead.

This resolves the most common case of multiple identical `FROM`
statements without adding more code to the frontends themselves.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-09-04 15:04:06 -05:00
Jonathan A. Sternberg 8841b2dfc8 dap: ensure test client is closed on cleanup
The dap test wasn't waiting for the client's goroutines to complete
before exiting which caused a race condition that could cause it to log
to the dead test logger. This became apparent when `--count` of greater
than one was used since it caused the test to run long enough to trigger
the behavior. It would have also triggered if we had added more tests.

Add the client close to the cleanup so it waits for the goroutine to
finish before the test exits as it was properly supposed to do.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-09-03 10:51:01 -05:00
Jonathan A. Sternberg a7c54da345 docs: update dap docs to reflect updates to the debugger
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-08-19 10:19:09 -05:00
Jonathan A. Sternberg 5c97696d64 dap: improve determination of the proper parent for certain ops
Improves the determination of the proper parent for exec and file ops.
With file ops, it will only consider inputs and ignore secondary inputs.
This prevents the following case:

```
FROM busybox AS build1
RUN echo foo > /hello

FROM scratch
COPY --from=build1 /hello .
```

Previously, `build1` would be considered the parent of the copy
instruction. Now, copy properly does not have a parent.

If there are multiple file ops and the operations disagree on the
canonical "parent", we give up on trying to find a canonical parent and
assume there is none.

For exec operations, whichever input is associated with the root mount
is considered the primary parent.

For all other operations, the first parent is considered the primary
parent if it exists.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-08-18 09:27:02 -05:00
Jonathan A. Sternberg b3c389690c dap: look for base name of dockerfile name instead of path from context
When the builder loads a dockerfile, it does it by using the base name
of the dockerfile path and only loads the innermost directory. This
means that the source name we're looking for is the base name and not
the full relative path.

Update the set breakpoints functionality so it takes this into account.
Fixes scenarios where DAP is used with a dockerfile nested in the
context.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-08-15 14:41:00 -05:00
Jonathan A. Sternberg dbda218489 dap: make exec shell persistent across the build
Invoking the shell will cause it to persist across the entire build and
to re-execute whenever the builder pauses at another location again.

This still requires using `exec` to launch the shell. Launching by frame
id is also removed since it no longer applies to this version.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-08-11 12:40:09 -05:00
Jonathan A. SternbergandGitHub 5c27294f27 Merge pull request #3327 from jsternberg/dap-fs-inspect
dap: filesystem inspection when paused on a digest
2025-08-05 11:06:16 -05:00
Jonathan A. Sternberg 8e356c3454 dap: filesystem inspection when paused on a digest
Add a file explorer to the debugger that allows exploring the filesystem
of the current container. It will show directory contents, file
contents, and symlink destinations. It will also show the file mode
associated with a file.

The file explorer defaults to marking itself as an expensive operation
so the debugger doesn't automatically retrieve the information.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-28 09:52:30 -05:00
Jonathan A. Sternberg 1e3c44709d dap: refactor how step in/step out works
Change how breakpoints and stepping works. These now work more how you
would expect another programming language to work. Breakpoints happen
before the step has been invoked rather than after which means you can
inspect the state before the command runs.

This has the advantage of being more intuitive for someone familiar with
other debuggers. The negative is that you can't run to after a certain
step as easily as you could before. Instead, you would run to that stage
and then use next to go to the step directly afterwards.

Step in and out also now have different behaviors. When a step has
multiple inputs, the inputs of non-zero index are considered like
"function calls". The most common cause of this is to use `COPY --from`
or a bind mount. Stepping into these will cause it to jump to the
beginning of the call chain for that branch. Using step out will exit
back to the location where step in was used.

This change also makes it so some steps may be invoked multiple times in
the callgraph if multiple steps depend on them. The reused steps will
still be cached, but you may end up stepping through more lines than the
previous implementation.

Stack traces now represent where these step in and step out areas
happen rather than the previous steps. This can help you know from where
a certain step is being used.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-23 17:10:40 -05:00
Jonathan A. Sternberg fea53ad1f8 dap: return error from evaluate command in repl context
In the repl context, we will now return the error instead of directly
printing it. We also suppress reporting errors from cobra. The logic
flow has also been changed to prevent returning errors from cobra unless
there was something related to the command line invocation so usage will
only be printed when a command was typed wrong and it will not show up
for every error.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-23 13:51:03 -05:00
Jonathan A. Sternberg ac9050261e docs: add docs related to dap
Adds some entry-level and developer-friendly docs for the debug adapter.
The one in `docs/dap.md` is meant for someone trying to use the debugger
while the one in `docs/reference/buildx_dap_build.md` is more focused on
documenting the command to be integrated in a debugger extension.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-15 10:58:01 -05:00
Jonathan A. Sternberg 3453f3b00a dap: support evaluate request to invoke a container
Supports using the `evaluate` request in REPL mode to start a container
with the `exec` command. Presently doesn't support any arguments.

This improves the dap server so it is capable of sending reverse
requests and receiving the response. It also adds a hidden command
`dap attach` that attaches to the socket created by `evaluate`.

This requires the client to support `runInTerminal`.

Likely needs some additional work to make sure resources are cleaned up
cleanly especially when the build is unpaused or terminated, but it
should work as a decent base.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-15 10:45:25 -05:00
Jonathan A. Sternberg e7b8de2b0c dap: correctly set the target when provided by the launch config
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-14 14:51:05 -05:00
Jonathan A. Sternberg e9d4b86161 dap: always return the error from execution if we paused on an error and resume
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-14 14:00:02 -05:00
Jonathan A. Sternberg 7925996c0c dap: do not modify variable references on variables that are zero
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-14 13:57:26 -05:00
Jonathan A. Sternberg 0a78659776 dap: alias step into to next and step out to continue
Step into and step out are required by the UI for DAP. We don't have a
way to implement these in a logical manner but they need to exist. We'll
discuss in further iterations how these might differ from next and
continue, but for now, we just need some implementation for the UI.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-14 11:00:33 -05:00
Jonathan A. Sternberg 1886e232c5 dap: implement variable references
Implement variable references to inspect the state of a stack frame.

Variable reference ids are composed of two sections. A thread mask that
is the first 8 bytes and the remainder is an increasing number that gets
reset each time a thread is resumed. This allows the adapter to know
which thread to delegate the variables request to and allows the
variable references to still remain confined to each thread. An int32 is
used for this because variable references need to be in the range of
(0, 2^32).

At the moment, only the platform variables and some of the exec
operations for an operation. These are labeled as "arguments" to the
stack frame.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-14 10:59:05 -05:00
Jonathan A. Sternberg 0dddf0a7b8 dap: implement first pass at breakpoints
Implement the first iteration of breakpoints. When a breakpoint is set,
it starts unverified. When a thread begins evaluation, it tries to see
if a breakpoint corresponds to one of the parsed instructions and will
verify it.

Breakpoints work when continue is used.

At the current moment, setting breakpoints while a thread is currently
running doesn't work. Breakpoints are rechecked each time execution is
about to restart.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-14 09:19:17 -05:00
Jonathan A. Sternberg a291698eaf dap: support stopOnEntry to configure behavior when starting the debugger
This will configure the default behavior when beginning to evaluate a
build target. When `stopOnEntry` is used, it will default to `stepNext`.
Otherwise, `stepContinue` will be used.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-07 09:39:23 -05:00
Jonathan A. SternbergandGitHub 3a3fc54e33 Merge pull request #3293 from jsternberg/dap-adapter-test-flaky
dap: increase timeout for receiving configuration done in adapter test
2025-07-03 09:47:42 -05:00
Jonathan A. Sternberg 4f2e23a9b8 dap: add stack traces with next and continue functionality
It is now possible to send next and continue as separate signals. When
executing a build, the debug adapter will divide the LLB graph into
regions. Each region corresponds to an uninterrupted chain of
instructions. It will also record which regions depend on which other
ones.

This determines the execution order and it also determines what the
stack traces look like.

When continue is used, we will attempt to evaluate the last leaf node
(the head). If we push next, we will determine which digest would be the
next one to be processed.

In the future, this will be used to also support breakpoints.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-03 09:18:55 -05:00
Jonathan A. Sternberg f03ed8cc9f dap: increase timeout for receiving configuration done in adapter test
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-07-03 09:12:34 -05:00
Jonathan A. Sternberg 42599a7d49 dap: add debug adapter implementation
Adds a simple implementation of the debug adapter that supports the very
basics of a debug adapter.

It supports the launch request, the configuration done request, the
creation of threads, stopping, resuming, and disconnecting from server.

It does not support custom breakpoints, stack traces, or variable
inspection yet. These are planned to be added in the future.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-30 10:51:20 -05:00
Jonathan A. Sternberg 3c2decea38 dockerfile: update buildkit to 0.23.0
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-17 13:29:22 -05:00
Jonathan A. Sternberg 7660acf9c7 progress: ensure bake waits for progress to finish printing on error conditions
Some minor fixes to the printer and how bake invokes it. Bake previously
had a race condition that could result in the display not updating on an
error condition, but it was much rarer because the channel communication
was much closer. The refactor added a proxy for the status channel so
there was more of an opportunity to surface the race condition.

When bake exits with an error when reading the bakefiles, it doesn't
wait for the printer to finish so it is possible for the printer to
update the display after an error is printed. This adds an extra `Wait`
in a defer to make sure the printer is finished.

`Wait` has also been fixed to allow it to be called multiple times and
have the same behavior. Previously, it only waited for the done channel
once so only the first wait would block.

The `onclose` method is now called every time the display is paused or
stopped. That was the previous behavior and it's been restored here.

The display only gets refreshed if we aren't exiting. There's no point
in initializing another display if we're about to exit.

The metric writer attached to the printer was erroneously removed. It is
now assigned properly.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-16 12:24:04 -05:00
Jonathan A. Sternberg 7f5ff6b797 commands: remove debug package in commands
The package just causes the entire flow to be more complicated as build
has to pretend it doesn't know about debug options and the debugger has
to pretend it doesn't know about the build.

This abstraction has been difficult when integrating a DAP command into
this same workflow so I don't think this abstraction has much of a
value.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-13 09:32:35 -05:00
Jonathan A. Sternberg e1adeee898 vendor: github.com/moby/buildkit v0.23.0-rc1
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-11 16:29:31 -05:00
Jonathan A. Sternberg 38cf84346c build: change build handler to evaluate instead of onresult
This changes the build handler to customize the behavior of evaluate
rather than onresult and also simplifies the `ResultHandle`. The
`ResultHandle` is now only valid within the gateway callback and can be
used to start containers from the handler.

`Evaluate` now executes inside of the gateway callback rather than
having a separate implementation that executes or re-invokes the build.
This keeps the gateway callback session open until the debugger has
returned.

The `ErrReload` for monitor has now been moved into the `build` package
and been renamed to `ErrRestart`. This is because it restarts the build
so the name makes a bit more sense. The actual use of this functionality
is still tied to the monitor reload.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-10 11:48:41 -05:00
Jonathan A. Sternberg 34e59ca1bd progress: fix progress writer pause and unpause to prevent panics
This changes the progress printer's pause and unpause implementation to
be reentrant to prevent race conditions and it also allows the status
updates to be buffered when the display is paused.

The previous implementation mixed the pause implementation with the
finish implementation and could cause a send on closed channel panic
because it could close the status channel before it had finished being
used. Now, the status channel is not closed.

When the display is enabled, the status channel will be forwarded to an
internal channel that is used to display the updates. When the display
is paused, the status channel will have the statuses buffered in memory
to be sent when the progress display is resumed.

The `Unpause` method has also been renamed to `Resume`.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-09 14:07:52 -05:00
Jonathan A. Sternberg 1d7cda1232 controller: remove remaining parts of the controller
Removes all references to the controller and moves the remaining
sections of code to other packages.

Processes has been moved to monitor where it is used and the data
structs have been removed so buildflags is used directly. The controller
build function has been moved to the commands package.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-05 11:57:03 -05:00
Jonathan A. Sternberg 8f2604b6b4 monitor: move remaining controller functionality into monitor
This creates a `Monitor` type that keeps the global state between
monitor invocations and allows the monitor to exist during the build so
it can be utilized for callbacks.

The result handler is now registered with the monitor during the build
and `Run` will use the result if it is present and the configuration
intends the monitor to be invoked with the given result.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-04 15:27:24 -05:00
Jonathan A. Sternberg 21ebf82c99 monitor: refactor how reload works
The build now happens in a loop and the monitor is run after every
build. The monitor can return `ErrReload` to signal to the main thread
that it should reload the build result.

This will be used in the future to move the monitor into a callback
rather than as a separate existence. It allows the monitor to not
control the build itself which now makes it possible to completely
remove the controller.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-04 15:06:31 -05:00
Jonathan A. Sternberg 65e46cc6af commands: simplify passing stdin to the build when the monitor is configured
The monitor needs stdin to run and isn't compatible with loading a
context or dockerfile from stdin. We already disallow this combination
and, with the removal of the remote controller, there's no way to use
stdin during the build when invoke is configured.

This just removes the extra code to allow forwarding stdin to the build
when the monitor is configured to simplify that section of code.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-03 15:41:23 -05:00
Jonathan A. Sternberg 6a0f5610e3 controller: remove the controller interface
The controller interface is removed and the local controller is used for
only the initial build, invoke, and rebuilds.

Process control has been moved to the monitor.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-03 15:41:23 -05:00
Jonathan A. Sternberg e78aa98c92 build: refactor some of the build functions into smaller utility functions
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-03 15:41:22 -05:00
Jonathan A. Sternberg 9bd1ba2f5c commands: update deprecation notice for keep-storage
The `--keep-storage` flag was changed to `--reserved-space`. Before it was
changed to that name, it was changed to `--max-storage`. This flag never
made it into a release as the name was changed before release, but the
update to the flag in buildx forgot to update the deprecation notice.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-06-02 09:35:39 -05:00
Jonathan A. Sternberg 781a3f117a hack: remove code generation related to generated files
With the removal of the protobuf for the controller, there are no longer
any generated files. Remove the makefile targets and the associated
dockerfiles and bake targets.

This wasn't being included in CI because it wasn't part of the
`validate` target.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-05-22 14:59:42 -05:00
Jonathan A. Sternberg 384f0565f5 controller: remove controller/errdefs protobuf files
Remove the protobuf files associated with controller/errdefs.

This doesn't completely remove the type as the monitor still uses it as
a signal to start the monitor.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-05-01 12:14:36 -05:00
Jonathan A. SternbergandGitHub ee77cdb175 Merge pull request #3102 from jsternberg/buildkit-rc1
vendor: github.com/moby/buildkit v0.21.0-rc1
2025-04-09 10:56:27 -05:00
Jonathan A. Sternberg 8fb1157b5f vendor: github.com/moby/buildkit v0.21.0-rc1
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-04-09 10:28:03 -05:00
Jonathan A. Sternberg 53e576b306 vendor: github.com/moby/buildkit v0.20.2
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-03-24 15:56:16 -05:00
Jonathan A. Sternberg e75ac22ba6 buildflags: skip empty cache entries when parsing
Broken in 11c84973ef. The section to skip
an empty input was accidentally removed when some code was refactored to
fix a separate issue.

This skips empty cache entries which allows disabling the `cache-from` and
`cache-to` entries from the command line overrides.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-02-24 10:09:02 -06:00
Jonathan A. Sternberg 75160643e1 ci: update buildkit to 0.20.0
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-02-19 15:21:14 -06:00
Jonathan A. SternbergandGitHub ad18ffc018 Merge pull request #3010 from jsternberg/vendor-update
vendor: github.com/moby/buildkit v0.20.0
2025-02-19 13:30:37 -06:00
Jonathan A. Sternberg 80c3832c94 vendor: github.com/moby/buildkit v0.20.0
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-02-19 13:17:40 -06:00
Jonathan A. SternbergandGitHub 7762ab2c38 Merge pull request #3008 from thaJeztah/bump_engine_28.0_rc3
vendor: github.com/docker/docker, docker/cli v28.0.0-rc.3
2025-02-19 11:59:57 -06:00
Jonathan A. Sternberg d6fdf83f45 bake: allow annotations to be set on the command line
Annotations were not merged correctly. The overrides in `ArrValue` would
be merged, but the section of code setting them from the command line
did not include `annotations` in the list of available attributes so the
command line option was completely discarded.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-02-14 11:57:30 -06:00
Jonathan A. Sternberg 3ae9970da5 buildflags: make work on go 1.22 by reverting rangefunc usage
Reverts the usage of rangefunc and attempts to keep the foundation of it
in for when we move to go 1.23. We have downstream dependencies that
aren't ready to move to go 1.23. We can likely move after go 1.24 is
released.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-02-10 11:03:46 -06:00
Jonathan A. Sternberg abc85c38f8 buildflags: handle unknown values from cty
Update the buildflags cty code to handle unknown values. When hcl
decodes a value with an invalid variable name, it appends a diagnostic
for the error and then returns an unknown value so it can continue
processing the file and finding more errors.

The iteration code has now been changed to use a rangefunc from go 1.23
and it skips empty or unknown values. Empty values are valid when they
are skipped and unknown values will have a diagnostic for itself.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-02-06 09:45:18 -06:00
Jonathan A. Sternberg 11c84973ef buildflags: fix ref only format for command line and bake
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-01-22 13:18:38 -06:00
Jonathan A. Sternberg 3aed658dc4 buildflags: marshal attestations into json with extra attributes correctly
`MarshalJSON` would not include the extra attributes because it iterated
over the target map rather than the source map.

Also fixes JSON unmarshaling for SSH and secrets. The intention was to
unmarshal into the struct, but `UnmarshalText` takes priority over the
default struct unmarshaling so it didn't work as intended.

Tests have been added for all marshaling and unmarshaling methods.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2025-01-21 15:05:23 -06:00
Jonathan A. SternbergandCrazyMax 1e992b295c bake: test empty override
Co-authored-by: CrazyMax <github@crazymax.dev>
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-12-18 11:56:19 -06:00
Jonathan A. Sternberg 4f81bcb5c8 bake: implement composable attributes for attestations
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-12-18 11:48:50 -06:00
Jonathan A. Sternberg 5dd4ae0335 bake: various fixes for composable attributes
This changes how the composable attributes are implemented and provides
various fixes to the first iteration.

Cache-from and cache-to now no longer print sensitive values that are
automatically added. These automatically added attributes are added when
the protobuf is created rather than at the time of parsing so they will
no longer be printed. If they are part of the original configuration
file, they will still be printed.

Empty strings will now be skipped. This was the original behavior and
composable attributes removed this functionality accidentally. This
functionality is now restored.

This also expands the available syntax that works with each of the
composable attributes. It is now possible to interleave the csv syntax
with the object syntax without any problems. The canonical form is still
the object syntax and variables are resolved according to that syntax.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-12-18 10:26:15 -06:00
Jonathan A. Sternberg 3ccbb88e6a bake: initial set of composable bake attributes
This allows using either the csv syntax or object syntax to specify
certain attributes.

This applies to the following fields:
- output
- cache-from
- cache-to
- secret
- ssh

There are still some remaining fields to translate. Specifically
ulimits, annotations, and attest.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-11-21 12:31:11 -06:00
Jonathan A. Sternberg a5bb117ff0 bake: improve error when using incorrect format for setting labels
Improves the error message when using an incorrect format for setting
labels. This includes the intended format directly in the error message
instead of assuming the user knows what the format is.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-11-04 14:38:23 -06:00
Jonathan A. Sternberg cf7a9aa084 pprof: take cpu and memory profiles by setting environment variables
When run in standalone mode, the environment variables
`DOCKER_BUILDX_CPU_PROFILE` and `DOCKER_BUILDX_MEM_PROFILE` will cause
profiles to be written by the CLI.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-10-24 09:56:27 -05:00
Jonathan A. Sternberg 64c5139ab6 hack: generate vtproto files for buildx
Integrates vtproto into buildx. The generated files dockerfile has been
modified to copy the buildkit equivalent file to ensure files are laid
out in the appropriate way for imports.

An import has also been included to change the grpc codec to the version
in buildkit that supports vtproto. This will allow buildx to utilize the
speed and memory improvements from that.

Also updates the gc control options for prune.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-10-08 13:35:06 -05:00