Commit Graph
100 Commits
Author SHA1 Message Date
Justin ChadwellandGitHub acc77888aa Merge pull request #3551 from jedevc/split-node-resolver 2025-12-05 08:14:23 +00:00
Justin Chadwell 3e413a05df build: split out node resolver
Signed-off-by: Justin Chadwell <me@jedevc.com>
2025-12-04 09:48:36 +00:00
Justin ChadwellandGitHub 1c7e6e7209 Merge pull request #3509 from jedevc/docs-syntax-build-context
docs: document using build contexts to override syntax directive
2025-11-13 13:59:24 +00:00
Justin Chadwell a516d3fb37 docs: document using build contexts to override syntax directive
Signed-off-by: Justin Chadwell <me@jedevc.com>
2025-11-11 16:15:47 +00:00
Justin ChadwellandGitHub 9ad116aa8e Merge pull request #2478 from thaJeztah/extract_resolve_digest
build: loadInputs: extract resolving digest to a separate function
2024-05-29 11:00:54 +01:00
Justin ChadwellandGitHub 714b85aaaf Merge pull request #2262 from Granjow/patch-1
Fix typo in URL
2024-02-20 14:45:18 +00:00
Justin ChadwellandGitHub 8f8ed68b61 Merge pull request #2250 from iankingori/add-npipe
driver: add npipe url scheme support
2024-02-07 13:46:01 +00:00
Justin ChadwellandGitHub 89684021b3 Merge pull request #2230 from jedevc/imagetools-resolver-copy-dupe
fix: avoid modifying source during resolver.Copy
2024-02-01 10:30:32 +00:00
Justin Chadwell 95bdecc145 fix: avoid modifying source during resolver.Copy
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-31 14:44:10 +00:00
Justin Chadwell aac7a47469 build: fix incorrect solve opt platform from being set
This regression was introduced in 616fb3e55c,
with the node resolution refactor.

The core issue here is just that we would unconditionally set the
solve opt's platform to the default current platform, which was
incorrect. We can prevent this easily by having a special case for the
default case, like we had before, and then not setting the platforms
field on this (which keeping the resolution behavior which was
introduced).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-28 10:11:51 +00:00
Justin Chadwell aa0aeac297 build: move solve opt out of duplicate map
This was more error prone, as opposed to the approach used prior to
616fb3e55c.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-28 10:10:50 +00:00
Justin ChadwellandCrazyMax 616fb3e55c build: rework node resolution
This patch reworks and updates the node resolution logic for selecting a
node from a builder.

The new implementation reworks the logic to make use of containerd's
platforms.Matcher interface instead of manually associated strings, and
additionally provides a few behavioural changes over the original
implementation, namely platforms can be matched with non-strict
semantics. e.g. i386 builds can be scheduled on an amd64 node, arm/v6
builds can be scheduled on an arm/v7 node.

We also add a new collection of tests for tracking regressions and
making the intended behaviour clearer.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-14 15:08:30 +01:00
Justin Chadwell d83da63320 bake: fix global target access when using a matrix
Previously, we would fail while trying to use the global "target" field
when using a matrix. The contents of the matrix really don't matter for
this.

What was happening was that we would copy the "target" property into the
child evaluation context, so that when it was updated on the parent, it
wouldn't propagate to the child. The correct behavior here is to avoid
copying variables from the target evaluation context if it is the
root.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-02 17:49:58 +00:00
Justin Chadwell adc839aa40 vendor: update buildkit to master@d5c1d785b042
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-10-23 14:48:50 +01:00
Justin ChadwellandGitHub 2c2294fa43 Merge pull request #2068 from crazy-max/inspect-fix-platforms 2023-09-30 09:25:02 +01:00
Justin Chadwell b9aad03e7a progress: create new display object after unpause
Without this, we attempt to overwrite the previous display in the lines
above which have been filled with logs from the monitor (see when used
with `--invoke`).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
Justin Chadwell 0bd6f3c7f5 remote: remove explicit backoff config from client opts
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
Justin Chadwell e2ebab5f26 vendor: update buildkit to master@cbfd4023383d
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
Justin ChadwellandGitHub 8d001e338f Merge pull request #2040 from crazy-max/validate-build-args 2023-09-11 16:51:20 +01:00
Justin ChadwellandGitHub 66e6dab26b Merge pull request #2020 from jedevc/build-annotation-flags 2023-09-11 15:55:01 +01:00
Justin Chadwell 0138f2a00f bake: add annotations field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:31:07 +01:00
Justin Chadwell a59058e8a5 build: add --annotation shortcut flag
This extracts the same logic for parsing annotations from the imagetools
create command, and allows the same flags to be attached to the build
command.

These annotations are then merged into all provided exporters.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:31:04 +01:00
Justin Chadwell 8fe2070d10 imagetools: make annotation parser more generic
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:30:31 +01:00
Justin Chadwell 54bb799d15 imagetools: simplify return type of annotation parser
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:30:31 +01:00
Justin ChadwellandGitHub fbf6594758 Merge pull request #2034 from tonistiigi/trace-delegate-driver 2023-09-06 11:20:56 +01:00
Justin ChadwellandGitHub 87e8e4b847 Merge pull request #2029 from testwill/loop
chore: slice loop replace
2023-08-31 07:10:10 +02:00
Justin ChadwellandGitHub da6662975f Merge pull request #2019 from jedevc/chore-bake-field-reorder 2023-08-24 14:17:55 +01:00
Justin Chadwell de4dbb7d00 chore: reorder target fields
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-24 14:01:14 +01:00
Justin ChadwellandGitHub 3bd4bca994 Merge pull request #2013 from jedevc/tests-add-imagetools-create
tests: add imagetools tests for copying manifests and indexes
2023-08-23 13:31:45 +01:00
Justin Chadwell 5941bf0494 tests: add imagetools tests for copying manifests and indexes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-22 17:38:36 +01:00
Justin Chadwell a43d9a67c7 git: fix error wrapping to ensure internal errors are propogated
Also, tidy up the error printing, so that now we always print out the
"current commit information was not captured by the build" message,
instead of just for not locating the git binary.

Before:

	WARNING: buildx: git was not found in the system. Current commit information was not captured by the build

After:

	WARNING: current commit information was not captured by the build: git was not found in the system: <error message>

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-22 12:17:17 +01:00
Justin Chadwell c47eb3bf5a git: propogate failure to locate git binary
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-22 12:16:19 +01:00
Justin Chadwell d37d483097 docker-container: avoid fail if container conflict
Fixes the race condition where two boots are executed simultaneously
across multiple processes.

We initially check to see if the container exists, but if during
container creation we get a name conflict, we don't treat this error as
a hard failure, and instead move immediately into waiting for the node
to boot.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-10 14:29:36 +01:00
Justin ChadwellandGitHub e5cee892ed Merge pull request #1965 from mqasimsarfraz/qasim/oci-annotations 2023-08-08 14:36:26 +01:00
Justin ChadwellandGitHub 51b9bab245 Merge pull request #1987 from jedevc/vendor-buildkit-master-tests 2023-08-07 17:08:16 +01:00
Justin ChadwellandGitHub 50fbdd86f9 Merge pull request #1985 from jsternberg/integration-test/version 2023-08-04 16:14:42 +01:00
Justin ChadwellandGitHub 1f61de0fcc Merge pull request #1988 from jedevc/fix-attests-on-docker-driver 2023-08-04 16:07:21 +01:00
Justin Chadwell e206c585bb build: error on attests on non-multiplatform driver
On drivers that do not support multi-platform builds (the default
`docker` driver), we do not support building attestations (unless using
the containerd store).

We need to check this feature before attempting to build using
attestations.

Also adds a test to ensure that attestations can be pushed to registries
at all, and that it adequately fails on the docker driver.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 15:28:32 +01:00
Justin Chadwell 5e46d8057d tests: add unsupported features detection skeleton
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 12:00:18 +01:00
Justin Chadwell 4e7709e54c vendor: update buildkit to master@b49a8873179b
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 12:00:18 +01:00
Justin Chadwell 5ed8f1b7d9 tests: avoid hardcoded driver check in testImageIDOutput
To detect if there is a docker daemon available, we can use the sandbox
.DockerAddress() function.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 11:22:10 +01:00
Justin ChadwellandGitHub 13ec635988 Merge pull request #1914 from ktock/updatedebugdocs 2023-08-03 09:52:13 +01:00
Justin ChadwellandGitHub d11dbbf9f7 Merge pull request #1978 from tonistiigi/imagetools-test 2023-07-31 15:39:42 +01:00
Justin ChadwellandGitHub a59fd3ebfe Merge pull request #1970 from ktock/entrypointconfig 2023-07-27 15:21:37 +01:00
Justin Chadwell 5a1f252bd9 progress: don't modify ResetTime inputs
No other parts of the progress rendering modify the inputs, so we should
avoid this as well.

This actually fixes an edge case in pushWithMoby which writes the same
VertexStatus multiple times, modifying the timestamps and similar.
However, if the operation takes long enough the small time difference
can accumulate, and move the Start time far into the past.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-25 09:35:43 +01:00
Justin ChadwellandGitHub b8739d7441 Merge pull request #1934 from jedevc/use-buildkit-client-wait 2023-07-19 16:23:54 +01:00
Justin ChadwellandGitHub 23fe02993b Merge pull request #1963 from jedevc/split-to-driver-pairs-fix-type 2023-07-19 15:57:43 +01:00
Justin Chadwell 1d177f00d2 chore: tidy splitToDriverPairs to avoid unneccessary int
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-19 14:43:50 +01:00
Justin Chadwell e11c5e3e96 remote: use buildkit's client.Wait method to bootstrap
This native implementation uses GRPC level waiting, instead of starting
a busy loop. We also a manual max backoff of one second to improve
responsiveness.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-19 12:39:19 +01:00
Justin ChadwellandGitHub e98e8f6ac9 Merge pull request #1948 from thaJeztah/buildkit_0.12 2023-07-18 15:05:18 +01:00
Justin ChadwellandGitHub 7c3b77fb36 Merge pull request #1950 from thaJeztah/remove_imageutil_dead_code 2023-07-17 12:26:05 +01:00
Justin ChadwellandGitHub 068382f5df Merge pull request #1936 from jedevc/hack-set-go-version-to-1.20.5 2023-07-12 14:49:59 +01:00
Justin Chadwell c4bec05466 hack: force go version to 1.20.5
A temporary workaround for "http: invalid Host header" introduced in
go 1.20.6.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-12 14:31:23 +01:00
Justin ChadwellandGitHub b84e0e11b4 Merge pull request #1918 from crazy-max/docs-hidden-fix 2023-07-03 11:50:04 +01:00
Justin ChadwellandGitHub d95f5f8f3b Merge pull request #1925 from dvdksn/test/build-progress 2023-07-03 10:48:52 +01:00
Justin ChadwellandGitHub cf9798cede Merge pull request #1919 from crazy-max/fix-build-details-link 2023-06-30 13:51:23 +01:00
Justin ChadwellandGitHub 2eba60db75 Merge pull request #1916 from jedevc/add-local-bake-test 2023-06-27 11:28:47 +01:00
Justin Chadwell 0dcbed3f53 tests: add simple local bake test
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-27 11:16:44 +01:00
Justin ChadwellandGitHub a4663b4b2e Merge pull request #1900 from cyphar/build-go_extra_flags 2023-06-21 09:59:19 +01:00
Justin ChadwellandGitHub 010e4c8d54 Merge pull request #1890 from jedevc/tests-share-docker-container-backend 2023-06-20 14:24:09 +01:00
Justin Chadwell 6f394a0691 tests: set a dedicated buildx config dir for each worker
This should help reduce any unexpected config conflict between workers.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:40:45 +01:00
Justin Chadwell efd7279118 ci: run docker-container tests in parallel
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:28:51 +01:00
Justin Chadwell 601056f3a7 tests: share single docker between docker-container backends
This means that we can run our docker-container tests in parallel again,
which can help speed up our test runs by a *significant* amount.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:28:49 +01:00
Justin Chadwell 0a7f96cbfb vendor: update buildkit to master@2d91ddcceedc
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:27:46 +01:00
Justin Chadwell 431732f5d1 bake: fix incorrect dockerfile resolution against cwd:// context
We need to resolve the strip the cwd:// prefix before attempting to
resolve the dockerfile. Otherwise, we'll get the cwd:// prefix in the
dockerfile name, which isn't stripped out later.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-14 12:18:53 +01:00
Justin Chadwell d4a2c8d0c3 dockerutil: cache feature collection
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-13 09:47:38 +01:00
Justin ChadwellandGitHub bef42b2441 Merge pull request #1876 from tonistiigi/remove-history-request 2023-06-13 09:38:05 +01:00
Justin ChadwellandGitHub 1f4ac09ffb Merge pull request #1874 from thaJeztah/bump_engine_24.0.2 2023-06-09 11:39:13 +01:00
Justin Chadwell f5f00e68ef bake(cli): allow passing in-stream using command.Cli
ReadLocalFiles should allow passing the stdin file as an argument, which
allows us to read from dockerCli.Stdin() to be consistent with other
commands in the same package.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:50:53 +01:00
Justin Chadwell 14aebe713e debug-shell(cli): allow passing in-stream using command.Cli
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:50:35 +01:00
Justin Chadwell 9d2388e6f5 vendor: update buildkit to master@67a08623b95a
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:36:52 +01:00
Justin ChadwellandGitHub 2c02db8db4 Merge pull request #1835 from ktock/long-form-json 2023-06-08 17:10:37 +01:00
Justin ChadwellandGitHub c30bcade2c Merge pull request #1870 from tianon/invoke-entrypoint-fixes 2023-06-08 16:03:24 +01:00
Justin ChadwellandGitHub 47e34f2684 Merge pull request #1813 from jedevc/oci-export-on-containerd 2023-06-07 15:19:54 +02:00
Justin ChadwellandGitHub 3d981be4ad Merge pull request #1850 from ktock/monitor-longhelp 2023-06-07 15:09:35 +02:00
Justin ChadwellandGitHub 696770d29c Merge pull request #1840 from jedevc/fix-check-path-for-bake-cmd-context 2023-06-06 17:49:04 +02:00
Justin Chadwell d34103b0d9 bake: fix potential context entitlements escape
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell c820350b5e tests: refactor cmd helpers to allow configuring cwd, etc
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell 61a7854659 tests: add bake test for remote subdir context
Fixed in 12b6a3ad9a, but now we have
regression tests! So we can add a check that we don't break this
behavior again.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell e859ebc12e tests: add bake test for remote cmd context override
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell ef997fd6d0 bake: fix BAKE_CMD_CONTEXT relative path resolution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell 76c96347ff tests: add basic remote bake context test
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell 48d7dafbd5 git: update gitutil test utilities
- Adds a new GitServeHTTP function to start an http server to serve a
  target git repository.
- Adds a new GitDir helper method to get the path to the .git
  directory
- Updates the GitAdd method to take a variable number of files

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell d03e93f6f1 test: tmpdir can be a test helper
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin ChadwellandGitHub fcb7810a38 Merge pull request #1854 from crazy-max/inspect-features 2023-06-06 16:59:58 +02:00
Justin ChadwellandGitHub 7cef021a8a Merge pull request #1804 from jedevc/fixup-solve 2023-06-06 09:12:10 +02:00
Justin Chadwell c6db4cf342 build: clarify NewResultHandle comment
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 09:08:30 +02:00
Justin ChadwellandGitHub a906149930 Merge pull request #1862 from jedevc/update-contributing-tests
docs: update CONTRIBUTING.md with specific test notes
2023-06-01 21:55:12 +01:00
Justin Chadwell af328fe413 docs: update CONTRIBUTING.md with specific test notes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-01 11:34:20 +01:00
Justin Chadwell 183a73abae build: docker exporter should instead use oci with containerd
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:59:55 +01:00
Justin Chadwell b7f0b3d763 build: clear exports for secondary solve request
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:56 +01:00
Justin Chadwell 5b27d5a9f6 build: cleanup res if returned in basic build
In practice, this shouldn't happen, but the check is good to include
anyways.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:03 +01:00
Justin Chadwell cd1648192e build: rename ResultContext to ResultHandle
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:50:46 +01:00
Justin Chadwell 8d822fb06c build: move main solve request into main gateway call
Now, we always perform the full solve request in the main gateway call.
This ensures that progress works properly, and makes the lifetime
semantics much clearer.

NewResultContext abstracts the details of a successful/failed build, to
always return a single ResultContext, even though the details of how a
gateway is created is different:
- For a failed build, we can just keep the gateway open.
- For a successful build, we immediately open another gateway and
  re-evaluate the build definition in that gateway. This should give an
  instant cache hit (since the build was just successful).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:46:46 +01:00
Justin ChadwellandGitHub 5e2f8bd64a Merge pull request #1853 from jedevc/logrus-show-error 2023-05-30 10:29:49 +01:00
Justin Chadwell 5788ab33d2 logutil: print error from WithError if found
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 10:00:24 +01:00
Justin Chadwell 6c62225d1b vendor: update buildkit to master@c36941f4a10e
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 09:10:52 +01:00
Justin Chadwell 2dae553d18 build: update outline fallback image
Subrequests have been included in docker/dockerfile:1.5 labs, so we can
update the fallback to point to this release.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
Justin Chadwell 91c17f25fb build: tidy up print func
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
Justin ChadwellandGitHub 354ccc9469 Merge pull request #1844 from tonistiigi/imageid-fixes 2023-05-26 10:21:53 +01:00