Conan recipe builder pattern

Conan Recipe Builder Pattern Link to heading

I work extensively with the Conan package manager. They have an extensive library of pre-made Conan recipes in their conan-center index, supporting many builds for common compilers (thanks to JFrog for the CI power). However, sometimes you need a different compiler version, a different compiler, or different options for your Conan recipe build.

Thanks to the Conan package tools and Conan itself, this is really easy to do. Because I have done this for many recipes, I’ve named this technique the Conan recipe builder pattern. The nice thing about this technique is that you can use standard CI jobs to generate your Conan recipe builds without writing your own conanfile.py and just use an existing recipe. This technique can be used for any Conan repository, not just the files from the conan-center index itself.

Building zlib Link to heading

For example, we could build “our” version of zlib. We create a typical git repo like a standard Conan recipe repo, but we only need a build.py file and not a conanfile.py, because we will fetch the file from the conan-center index master branch. Install Conan and the Conan package tools via pip install conan conan-package-tools if not already done.

Here is the build.py file for the zlib builder:

import os
import shutil
from cpt.packager import ConanMultiPackager
from conans import tools

def create_clean_build_folder(folder):
    if os.path.exists(folder):
        shutil.rmtree(folder)
    os.makedirs(folder)

if __name__ == "__main__":
    url = "https://github.com/conan-io/conan-center-index/archive/master.zip"
    recipe_folder = "conan-center-index-master/recipes/zlib/1.2.8"
    build_folder = "_build"
    reference = "zlib/1.2.8@"

    create_clean_build_folder(build_folder)

    # Download the conan index as a zip file and extract to a subdirectory
    tools.get(
        url,
        filename="conan-index.zip",
        destination=build_folder,
        pattern=os.path.join(recipe_folder, "*"), # Extract only zlib/1.2.8
        retry=2,
        retry_wait=5
    )
    conanfile = os.path.join(build_folder, recipe_folder, "conanfile.py")

    # Normal CPT build, but with the downloaded conanfile
    builder = ConanMultiPackager(conanfile=conanfile, reference=reference)
    builder.add_common_builds()
    builder.run()

It looks like a standard build.py file, but we download the conanfile.py from the index. We create a local directory (_build) and download and unzip the file via tools.get (using the pattern variable to extract only the zlib/1.2.8 subdirectory). This is a method from Conan typically used in a conanfile.py file, but Conan is a standard Python package, so we can use it outside of a conanfile.py.

At the end, we perform a normal Conan package tools build, but we define the conanfile parameter (pointing to the downloaded conanfile from the conan-center index). You can use any parameter as defined in the Conan package tools repository and the Conan recipe itself.

Conclusion Link to heading

As you can see, it is really easy to compile custom builds from existing Conan recipes. Combine this technique with your CI tool of choice, and you can achieve automated builds as well.