HDL containers

Ready-to-use images are provided for each tool, which contain the tool and the dependencies for it to run successfully. These are typically named hdlc/<TOOL_NAME>. Since all of them are based on the same root image, pulling multiple images involves retrieving a few additional layers only. Therefore, this is the recommended approach for CI or other environments with limited resources.

Multiple tools from fine-grained images are included in larger images for common use cases. These are named hdlc/<MAIN_USAGE>. This is the recommended approach for users who are less familiar with containers and want a quick replacement for full-featured virtual machines. Coherently, some common Unix tools (such as make or cmake) are also included in these all-in-one imags.

By default, tools with Graphical User Interface (GUI) cannot be used in containers, because there is no graphical server. However, there are multiple alternatives for making an X11 or Wayland server visible to the container. mviereck/x11docker and mviereck/runx are full-featured helper scripts for setting up the environment and running GUI applications and desktop environments in OCI containers. GNU/Linux and Windows hosts are supported, and security related options are provided (such as cookie authentication). Users of GTKWave, nextpnr and other EDA tools will likely want to try x11docker (and runx).

USB/IP protocol allows to pass USB device(s) from server(s) to client(s) over the network. As explained at kernel.org/doc/readme/tools-usb-usbip-README, on GNU/Linux, USB/IP is implemented as a few kernel modules with companion userspace tools. However, the default underlying Hyper-V VM machine (based on Alpine Linux) shipped with Docker Desktop (aka docker-for-win/docker-for-mac) does not include the required kernel modules. Fortunately, privileged docker containers allow to install missing kernel modules. The shell script in usbip/ supports customising the native VM in Docker Desktop for adding USB over IP support.

Understanding how all the pieces in this project fit together might be daunting for newcomers. Fortunately, there is a map for helping maintainers and contributors traveling through the ecosystem. Subdir graph/ contains the sources of directed graphs, where the relations between workflows, dockerfiles, images and tests are shown.

(Graphviz)'s digraph format is used, hence, graphs can be rendered to multiple image formats. The SVG output is shown in Figure 1 describes which images are created in each map. See the details in the figure corresponding to the name of the subgraph: Base (Figure 2), Sim (Figure 3), Synth (Figure 4), Impl (Figure 5), Formal (Figure 6). Multiple colours and arrow types are used for describing different dependency types. All of those are explained in the legend: Figure 7.

Each tool/project is built once only in this image/container ecosystem. However, some (many) of the tools need to be included in multiple images for different purposes. Moreover, it is desirable to keep build recipes separated, in order to better understand the dependencies of each tool/project. Therefore, hdlc/pkg:* images are created/used (coloured BLUE in the [Graph]). These are all based on scratch and are not runnable. Instead, they contain pre-built artifacts, to be then added into other images through COPY --from=.

Since hdlc/pkg:* images are not runnable per se, but an intermediate utility, the usage of environment variables PREFIX and DESTDIR in the dockerfiles might be misleading. All the tools in the ecosystem are expected to be installed into /usr/local, the standard location for user built tools in most GNU/Linux distributions. Hence:

Despite the usage of these variables being documented in GNU Coding Standards, DESTDIR seems not to be very used, except by packagers. As a result, contributors might need to patch the build scripts upstream. Sometimes DESTDIR is not supported at all, or it is supported but some lines in the makefiles are missing it. Do not hesitate to reach for help through the issues or the chat!

A helper script is provided in .github/bin/ to ease building images, by hiding all common options:

All ready-to-use images (coloured GREEN or BROWN in the [Graph]) are runnable. Therefore, users/contributors can run containers and test the tools interactively or through scripting. However, since hdlc/pkg images are not runnable, creating another image is required in order to inspect their content from a container. For instance:

In fact, .github/bin/dockerTestPkg uses a similar dockerfile for running *.pkg.sh scripts from test/. See Test.

Alternatively, or as a complement, wagoodman/dive is a lightweight tool with a nice terminal based GUI for exploring layers and contents of container images. It can be downloaded as a tarball/zipfile, or used as a container:

There is a test script in test/ for each image in this ecosystem, according to the following convention:

Furthermore, hdl/smoke-test is a submodule of this repository (test/smoke-test). Smoke-tests contains fine grained tests that cover the most important functionalities of the tools. Those are used in other packaging projects too. Therefore, container tests are expected to execute the smoke-tests corresponding to the tools available in the image, before executing more specific tests.

There are a couple of helper scripts in .github/bin/, for testing the images. Those are used in CI but can be useful locally too: