bedrock.hcl

This section provides an overview on how to write the hcl configuration file.

What is .hcl file?

bedrock.hcl is an essential configuration file for Bedrock to work with your Git repo. It is a blueprint written by the user to tell Bedrock how to run training/ batch scoring pipeline or serving model as an endpoint. HCL is a JSON-compatible configuration language that we use to specify the blueprint.

bedrock.hcl format

bedrock.hcl can have three types of stanzas namely train, batch_score and serve.

A stanza tells Bedrock about how to run the code specified for the respective stage of the machine learning workflow. For instance, train stanza is used to tell Bedrock about what is required and used to run the training code. The explanations of each option available in different stanzas are explained below:

Components of a stanza

Generally, bedrock.hcl comprises the following:

  • train: the stanza where a training pipeline is defined

  • batch_score: the stanza where a batch scoring pipeline is defined

  • serve: the stanza where your serving script to deploy your model is placed

  • step: the basic unit of train and batch_score stanzas.

  • image: the base Docker image that the script will run in

  • install: the command to install any other packages not covered in the image

  • script: the command that calls the script

  • depends_on: a list of the names of steps that a step depends on

  • resources: computational resources of a step, including cpu, memory and gpu.

  • parameters: any environment variables used by the script for convenience

  • secrets: the names of the secrets necessary to run the script successfully

train ,batch_score and servestanzas all have parameters and secrets stanzas. step stanzas are only defined in train and batch_score stanzas. On the other hand, image, install and scriptshould also be defined in theserve stanza.

step is the basic unit of train and batch_score stanzas. It defines a task of a pipeline. Each step stanza has image, install ,script and depends_on. The name of each step can only contain numbers, alphabets, underscores and hyphens. It must begin and end in numbers or alphabets, and the name of each step must be unique.

image refers to the Docker image that your steps will run in. You can use your own image as the base image. The only requirement is that the image must have sh.

We have provided a standard Docker base image basisai/workload-standard for you to get started. It consists of Python 3.7, GCS connector and Spark jars. Kubernetes has some specific requirements about Spark, so we provide this image to demonstrate how to install Spark on Kubernetes.

If there are other specific packages to be installed for each step and are not included in the base image, you can add them to your requirements-[train/batch/serve].txt and install them using install.option.

script refers to the commands or files containing the code for the corresponding step.

resourcesdefines the computational resources requested for a step. When writing the hcl file, these rules must be followed:

  • cpu: Cores available in a step. Integers, fractional numbers and "millicores" are allowed for this field. For example, 1, 0.5, 500mare all valid, and 0.5 is identical to 500m. Please note that precision finer than 1m is not allowed.

  • memory: Memory available in a step. Memory requests are measured in bytes. You can express memory as a plain integer or as a fixed-point integer using one of these suffixes: E, P, T, G, M, K. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi, Mi, Ki. For example, 128974848, 129e6, 129M, 123Mi are all valid and represent roughly the same value.

  • gpu: GPU cores available in a step if GPU is supported. Only integers are allowed for this field.

For detailed information, please refer to this page.

parameters can be used to parametrise any environment variables used in any part of your code. In train and batch_score stanzas, these parameters will be shared among all steps. You can use this functionality for a wide variety of use cases such as model hyperparameters, input data tables or any other data metadata related to training run. These parameters will also be shown on Bedrock web UI for your reference.

For Python users, to stream logs produced by your code smoothly, we setPYTHONUNBUFFERED = 1by default. If you want to turn this off, please addPYTHONUNBUFFERED = ""to parameters in your.hclfile.

secrets can be used to name all secrets such as database secrets, which are needed to run the script successfully. Secrets in Bedrock are backed by a specialised secrets database Vault to protect the secrets.

Only provide the names of the secrets in the stanza, and NOT the secret values (e.g. passwords). You will only provide the secret values when you trigger a pipeline run from Bedrock web UI "Run pipeline" page. For example, if your database is protected by an access key with secret key (like7MD89NGhfd45hxRf76iCYEXAMPLEKEY), just supply the name of the access key (say SECRET_KEY) in the stanza:

secrets = [
"SECRET_KEY"
]

and enter7MD89NGhfd45hxRf76iCYEXAMPLEKEY to SECRET_KEY on Bedrock web UI "Run pipeline" page. In this way, you do not have to commit any secrets to GitHub and neither will they be shown on Bedrock UI.

train stanza

train stanza tells Bedrock how to run your training pipeline. We shall show two different kinds of train stanzas below for:

  • non-Spark use cases, and

  • Spark use cases.

Train stanza for non-Spark use cases

We define our task in a step inside the train stanza. The step stanza for non-Spark use cases is more straight-forward. In the following example, we define a step called "my-task":

train {
step "my-task" {
image = "basisai/workload-standard"
install = ["pip3 install -r requirements-train.txt"]
script = [
{
sh = ["python3 train.py"]
}
]
resources {
cpu = "500m"
memory = "500M"
}
}
parameters {
my_parameter_1 = "my_string_1"
my_parameter_2 = "my_string_2"
}
secrets = [
"DUMMY_SECRET_A",
"DUMMY_SECRET_B"
]
}

Train stanza for Spark use cases

If you wish to use Spark, you will need to set the script option of the step stanza intrain stanza with a spark-submit that includes three things:

  • [required] script: this will be train.py,

  • [required] conf: Spark configuration,

  • [optional] settings: any jars and zip files to be sent to all workers.

An example of such train stanza is shown below:

GCP
AWS
GCP
train {
step "spark" {
image = "basisai/workload-standard"
install = ["pip3 install -r requirements-train.txt"]
script = [
{spark-submit {
script = "train.py"
// to be passed in as --conf key=value
conf {
spark.kubernetes.container.image = "basisai/workload-standard"
spark.kubernetes.pyspark.pythonVersion = "3"
spark.driver.memory = "4g"
spark.driver.cores = "2"
spark.executor.instances = "2"
spark.executor.memory = "4g"
spark.executor.cores = "2"
spark.memory.fraction = "0.5"
spark.sql.parquet.compression.codec = "gzip"
spark.hadoop.fs.AbstractFileSystem.gs.impl = "com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS"
spark.hadoop.google.cloud.auth.service.account.enable = "true"
}
// to be passed in as --key=value
settings {
py-files = "my_zip_files.zip"
// If you want to load data from BigQuery
jars = "gs://spark-lib/bigquery/spark-bigquery-latest.jar"
}
}}
]
resources {
cpu = "0.5"
memory = "1G"
}
}
parameters {
my_parameter_1 = "my_string_1"
my_parameter_2 = "my_string_2"
}
secrets = [
"DUMMY_SECRET_A",
"DUMMY_SECRET_B"
]
}
AWS
train {
step "my_step_1" {
image = "basisai/workload-standard"
install = ["pip3 install -r requirements-train.txt"]
script = [
{spark-submit {
script = "train.py"
// to be passed in as --conf key=value
conf {
spark.kubernetes.container.image = "basisai/workload-standard"
spark.kubernetes.pyspark.pythonVersion = "3"
spark.driver.memory = "4g"
spark.driver.cores = "2"
spark.executor.instances = "2"
spark.executor.memory = "4g"
spark.executor.cores = "2"
spark.memory.fraction = "0.5"
spark.sql.parquet.compression.codec = "gzip"
spark.hadoop.fs.s3a.impl = "org.apache.hadoop.fs.s3a.S3AFileSystem"
}
// to be passed in as --key=value
settings {
py-files = "my_zip_files.zip"
jars = "my_jar_file_1.jar,my_jar_file_2.jar"
}
}}
]
}
parameters {
my_parameter_1 = "my_string_1"
my_parameter_2 = "my_string_2"
}
secrets = [
"DUMMY_SECRET_A",
"DUMMY_SECRET_B"
]
}

You can find the configurations specific to Spark on Kubernetes here.

batch_score stanza

batch_score stanza tells Bedrock how to run your batch scoring pipeline. This stanza is exactly similar to train stanza in structure and you can follow the same guidelines to write your batch_score stanza, including the Spark configurations if you are using Spark.

batch_score {
step "my-batch-scoring-job" {
image = "basisai/workload-standard"
install = ["pip3 install -r requirements-batch_score.txt"]
script = [{sh = ["python3 batch_score.py"]}]
resources {
cpu = "0.5"
memory = "1G"
}
}
parameters {
my_parameter_3 = "my_string_3"
my_paramter_4 = "my_String_4"
}
secrets = [
"DUMMY_SECRET_D",
"DUMMY_SECRET_E"
]
}

serve stanza

serve stanza tells Bedrock how to run your serving script to deploy your model as an API endpoint. Unlike train and batch_score stanzas, you only need to include the following elements:

  • [required] image: the base Docker image that the script will run in

  • [optional] install: the command to install any other packages not covered in the image

  • [required] script: the command that calls the script

HTTP
gRPC
HTTP
serve {
image = "python:3.7"
install = ["pip3 install -r requirements-serve.txt"]
script = [{sh = ["gunicorn --bind=:${SERVER_PORT} --worker-class=gthread --timeout=300 serve_http:app"]}]
}
gRPC
serve {
image = "python:3.7"
install = [
"pip3 install -r requirements-serve.txt",
"python3 -m grpc_tools.protoc -I protos --python_out=. --grpc_python_out=. protos/serve.proto"
]
script = [{sh = ["python3 serve_grpc.py"]}]
}

Note that the image used here is python:3.7 since it is most unlikely that Spark is used and so a Python image suffices.

You would also have noticed that for gRPC, there is an additional line in install:

python3 -m grpc_tools.protoc -I protos --python_out=. --grpc_python_out=. protos/serve.proto

This is to generate the special client and server code serve_pb2 and serve_pb2_grpc, which you might have observed in the imports for gRPC serving script. These two files are generated using the protocol buffer compiler protoc as well as the proto file serve.proto. Please see our "Writing files for Bedrock" guide on how to write serve.py and its proto file .

But if you have already generated these two Python files and save them in protos/, then it is not necessary to include that line in install.