Home | News | Upgrading the Clash compiler to a new version of GHC

Upgrading the Clash compiler to a new version of GHC

Christiaan Baaij, PhD

Table of contents

+
-

  1. Upgrading clash-prelude
  2. Upgrading clash-lib
  3. Upgrading clash-ghc
  4. Upgrading our continuous integration framework
  5. Closing words

Every so often the team behind the Glasgow Haskell Compiler (GHC) releases a new version that comes with a ton of new features. As the Clash compiler is closely linked with GHC, we need to upgrade the code for the Clash compiler so that we can compile it against the latest version of GHC in order to use those new features. This post describes the changes we had to make to compile Clash against GHC 9.10, resulting in the following PR: https://github.com/clash-lang/clash-compiler/pull/2758.

Upgrading clash-prelude

We start with the Clash base library for circuit description, clash-prelude, which is really just a regular Haskell library. Aside from raising the upper limits on the libraries we depend on, upgrading clash-prelude involved three aspects this time:

  1. In order to compile the code warning-free, we have to conditionally import foldl’ on GHC versions older than GHC 9.10. That’s because foldl’ is now exported by default from the standard Haskell Prelude as of GHC 9.10.
  2. The dataToTag# primitive function is no longer exported from GHC.Prim, as it is no longer a primitive function. It has been replaced by two new primitives: dataToTagSmall# and dataToTagLarge#. However, you are not supposed to use those two functions directly. The dataToTag# function has however always been exported from GHC.Exts, so all that needs to happen is to import it from that module, and everything will compile against GHC version 9.10 and against older versions of GHC.
  3. We use doctest-parallel to ensure that the output of the code examples in our documentation actually matches the output you would see when you run the expression in the REPL. Some of our code examples actually show the error message when an expression is either ill-typed, or given an input that results in a run-time error. Prior to GHC 9.10 there would be a blank line between the expression typed at the REPL and the error message; however, as of GHC 9.10 that blank line is no longer between the failing expression and the error message. Instead, there is now a blank line between the error message and the new prompt of the REPL. We basically moved from a leading ‘blank line’ to a trailing ‘blank line’. This meant we have to adjust all our examples using CPP to differentiate whether we are compiling with GHC 9.10 or with an older version.

Upgrading clash-lib

The clash-lib library forms the heart of the Clash compiler. It contains the code that:

  • Rewrites the Core (a “mini-Haskell”) description of a Haskell/Clash description to a version which can be “trivially” converted to a circuit netlist.
  • Performs the VHDL/(System)Verilog generation from such a rewritten Core description.
  • Handles the capability of the Clash compiler to add new primitives from user code.

While we try to have clash-lib as independent from the internals of GHC as possible, possibly allowing for a different front-end, there are parts where we need to agree with the internals of GHC. One such part is the identification of certain built-in/primitive type families, such as type-level addition of natural numbers. Part of upgrading clash-lib to GHC 9.10 involved the identification of these built-in/primitive identifiers:

  • As of GHC 9.10, the internals of GHC use Word64 to uniquely identify things, while clash-lib uses Int, just like GHC used to do before the release of 9.10. GHC changed the representation to Word64, as Int proved insufficient to represent all uniques on 32-bit platforms. Moving clash-lib to Word64 would involve a lot of changes we did not want to make as part of the upgrade to GHC 9.10. Instead, we use fromIntegral as a stop-gap measure at all the places we interact with the uniques that we get from GHC. For 64-bit platforms, which we assume nearly all Clash users use, this should not cause any issues. We will move to Word64 for our uniques at a later stage as we rethink some other aspects of our identifier-related code.
  • GHC is now shipped with a ghc-internal library which makes it easier to decouple the base library from GHC releases. As a result some built-in/primitive identifiers got moved from the GHC. to the GHC.Internal. module hierarchy, while other got moved from the GHC.Internal. hierarchy to the GHC. hierarchy. As parts of the clash-lib code base still uses string/text-based identication of GHC built-in/primitive identifiers, those parts had to be extended with the new fully qualified names.

Upgrading clash-ghc

The clash-ghc package provides:

  • Utilities to load Haskell modules using the GHC API and get GHC’s version of the Core representation of those modules
  • The conversion from GHC’s Core to Clash’s Core.
  • The clash and clashi executables, which are near exact copies of the ghc and ghci binaries, except that they are extended to use the above two bullet points in combination with clash-lib to also convert Haskell to HDL.

Upgrading clash-ghc always includes at least one very mechanical process:

  1. We copy the source of the ghc binary into a directory that will be used when the clash binary is compiled with GHC 9.10. We have such a directory in the source-tree of the clash-ghc package for every major version of GHC supported by Clash.
  2. We create a .patch of a previous upgrade of clash-ghc, limited only to those parts related to the ghc binary, and try to apply it to the copy from step 1. If the patch fails to apply cleanly, we simply use it as a guide for a more manual approach to extend the ghc binary with the Clash features.

Aside from this somewhat mechanical process, the upgrade required similar changes as those for clash-lib and were related to uniques being Word64 as of GHC 9.10.

Upgrading our continuous integration framework

While technically not part of our upgrade to GHC 9.10, we hadn’t actually added testing against GHC 9.8 to our continuous integration (CI) framework before starting on the GHC 9.10 upgrade. That is because when we added support for GHC 9.8 there wasn’t an official release of it yet. As part of our CI framework upgrade, we now use cabal-install version 3.12, which did require some interesting changes:

  1. As part of our CI, we want to test against the latest versions of our dependencies (within the version limits that we set). That way we can catch problems we might have with our dependencies without having to constantly update our dependencies in our working tree while we are developing. While cabal-install allows you to track the latest index-state of Hackage by setting it to HEAD, we set it to the date and time that we start a CI job. This way, we can split building and testing across different sub-jobs without the risk that we do rebuilding during the testing phase because our dependencies have had a new version uploaded to Hackage in the meantime.
    However, cabal-install comes 3.12 comes with https://github.com/haskell/cabal/pull/8944, which rejects any index-state that is newer than the last known index state. However, after a cabal update, you do not get an index-state that is basically the equivalent now, instead you get the date and time of when Hackage was actually last updated and indexed. Consequently, we now have to parse the date and time that gets reported by the cabal update process and set that as our index-state.
  2. Our dependencies will most likely not get updated every time we run a CI job, so we want to cache the Cabal store which holds the compiled versions of our dependencies. Before Cabal version 3.12, this store would by default be located at $CABAL_HOME/store/ghc-$GHC_VERSION. As of version 3.12, however, Cabal includes https://github.com/haskell/cabal/pull/9618, which results in the store being located at the moral equivalent of $CABAL_HOME/store/ghc-$GHC_VERSION-$GHC_ABI_HASH. This change was made so that ABI-incompatible variants of the same GHC version do not corrupt the store. As a result, we had to change our CI framework to ghc-pkg recache from a store path that includes the $GHC_ABI_HASH when the version of GHC actually reports it; which is GHC version 9.8 and up.

Closing words

Being closely linked to GHC, upgrading Clash to a new version of GHC is on average more involved than it would be for the average Haskell project. The changes required to compile against version 9.10 of GHC were relatively minor; especially compared to earlier upgrades which involved changes with regards to interacting with the GHC API. To be fair though, that is also because we punted updating Clash’s unique representation to Word64, to be in line with the unique representation of GHC 9.10. As a result, the Clash compiler might give unreliable or even buggy results on 32-bit platforms when compiled against GHC 9.10. As a work-around, users on 32-bit platforms would have to compile against GHC 9.8 or earlier for now. Also, this blog post is written before a new version of the Clash compiler with the GHC 9.10 support is actually released. It might very well be that the unique issue will be resolved before a released version can be found on Hackage.

Would you like to share this article?

Related articles