Continuous Learning

/home

/posts

Bazel build system and Rust

I will write more posts for each step I’m learning Bazel together with Rust, but don’t wait for the next post because that could take a while.

A build system is a tool for automating the build process of software. There are multiple well known build systems which have been used for decades, make is one of the older ones. Then we have CMake, MSBuild, Gradle, Ant, Maven and the list can go on.

What is common for all build systems is: They have a process for how to compile source code into binary code, packaging binary code, and running automated tests.

Most common way to use build systems is to use a continuous integration chain which is activated when there is a committed change in the source or a new release.

But still developers need to be able to build and execute their code locally on their desktops for shorter round trip to have a binary to test against in their environment.

Bazels way of defining the builds enables that both the continuous integration chain and the developers local compiling process can take advantage of each other by having a shared cache of all builds and its dependencies. Developers have multiple iterations and changing their code and then compile it until they are satisfied, all these changes will have their own build which can be tracked and cached. So when the developer commit their code to the central repository, the build cache is also shared with the continuous integration chain so it can jump directly to testing without the need to rebuild everything.

This enables so a developer can pull the whole source code and build it without rebuilding, because everything is in the cache.

Why not cargo #

Cargo is the build system and package manager in Rust and is well used by the rust community. It is the de facto standard in the Rust sphere. And by that it is easy to have a package system for different libraries or crates which they are called in Rust.

So why not continue to use Cargo, you would first start argue with.

For most of the projects out there, Cargo will be sufficient and there is no need to use Bazel or other build systems than Cargo.

But the moment you have multiple teams in charge of different software components which shall work together in a bigger system, Cargo gets out grown pretty quickly. It can also be the case that different parts of the system is written in different programming languages. Bazel can handle different programming languages with rules, which is a set of actions which Bazel need to perform for making a build for that specific programming language or toolchain.

A simple example #

The following example is a minimal example of a simple rust project consisting of a binary, library and external crates. This can be handled by Bazel.

You can view and clone the following project from github Anders-Linden / rust_bazel_hello_world

git clone <https://github.com/Anders-Linden/rust_bazel_hello_world.git>

or just continue to follow bellow and manually set up the project.

Folder structure #

mkdir -p hello_world/srcmkdir -p hello_world/utilities/srcmkdir hello_world/assets

Workspace #

The Workspace file is the place Bazel is considering as the root path of the workspace, in this file it is stated what rules this project is going to be built by.

This example uses the rust rules

cat <<EOF > hello_world/WORKSPACE.bazel
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "io_bazel_rules_rust",
    sha256 = "618cba29165b7a893960de7bc48510b0fb182b21a4286e1d3dbacfef89ace906",
    strip_prefix = "rules_rust-5998baf9016eca24fafbad60e15f4125dd1c5f46",
    urls = [
        # Master branch as of 2020-09-24
        "<https://github.com/bazelbuild/rules_rust/archive/5998baf9016eca24fafbad60e15f4125dd1c5f46.tar.gz>",
    ],
)

load("@io_bazel_rules_rust//rust:repositories.bzl", "rust_repositories")

rust_repositories(version = "1.47.0", edition="2018")

load("@io_bazel_rules_rust//:workspace.bzl", "rust_workspace")

rust_workspace()

load("//cargo:crates.bzl", "hello_cargo_library_fetch_remote_crates")

hello_cargo_library_fetch_remote_crates()
EOF

Build files #

For each binary or library or other artifact there is a BUILD file, this defines the source files and its dependendencies.

Binary build file # 

cat <<EOF > hello_world/BUILD.bazel
load("@io_bazel_rules_rust//rust:rust.bzl", "rust_binary")

rust_binary(
    name = "hello_world",
    srcs = ["src/main.rs"],
    data = ["assets/hello_world.txt"],
    deps = ["//utilities",
            "//cargo:log",
            "//cargo:env_logger"],
)
EOF

Library build file #

A library is also built and then linked to a binary and the same code can be used in multiple binaries.

cat <<EOF > hello_world/utilities/BUILD.bazel
package(default_visibility = ["//visibility:public"])

load("@io_bazel_rules_rust//rust:rust.bzl", "rust_library")

rust_library(
    name = "utilities",
    edition = "2018",
    srcs = [
        "src/lib.rs",
    ],
)
EOF

Source files #

This is ordinary Rust source files for the project

Library source # 

cat <<EOF > hello_world/utilities/src/lib.rs
use std::fs::File;
use std::io::BufReader;

pub fn open_input(file_path: &str) -> std::io::BufReader<std::fs::File> {
	let file = match File::open(file_path) {
		Err(why) => panic!("couldn't open, {}", why),
		Ok(file) => file,
	};
	BufReader::new(file)
}
EOF

Binary source # 

cat <<EOF > hello_world/src/main.rs
use std::io::prelude::*;
extern crate utilities as utils;

use log;

fn main()  {
    env_logger::init();
    log::info!("Starting");
	for line in utils::open_input("./assets/hello_world.txt").lines() {
        log::info!("{}", line.unwrap());
	}
}
EOF

Asset file #

Just a simple text-file which is going to be read and parsed.

cat <<EOF > hello_world/assets/hello_world.txt
hello, this is first row
and this is the second row
EOF

Cargo file #

The cargo file is used by cargo-raze for generating an environment and the BUILD files for each external crate.

Cargo-raze have different ways of handling crates

  • Vendoring mode
  • Remote Dependency Mode

And then there is different ways of handling the building of crates depending on the complexity of the crate.

  • Simple crates
  • Unconventional Crates
  • Crates that need system libraries
  • Crates that supply useful binaries
  • Crates that only provide binaries

Read the documentation on Github cargo-raze

cat <<EOF > hello_world/Cargo.toml
[package]
name = "hello_world"
version = "0.0.1"
edition = "2018"

[dependencies]
log = {version = "0.4.11", features = ["std"]}
env_logger = "0.8.1"

[[bin]]
name = "hello_world"
path = "src/main.rs"

[raze]
workspace_path = "//cargo"
target = "x86_64-apple-darwin"
output_buildfile_suffix = "BUILD.bazel"
gen_workspace_prefix = "hello_cargo_library"
genmode = "Remote"
default_gen_buildrs = true
EOF

Go inside the workspace path

cd hello_world

The cargo lockfile consist of the exact information for each dependency which is used in the rust project

cargo generate-lockfile

The command for generating the Bazel Build files for external crates

cargo raze

Command for building and running the application.

bazel run :hello_world
←
Git rebase when you understand why
back to top