Work with files in Terraform
Here are two handy functions to interact with files in Terraform.
Terraform supports file system and we can read file contents and parse to an attribute in order to organize files in the better way.
Here are two handy functions to interact with files in Terraform.
file()
The function file() in Terraform allows a single file to be read as a plain string.
For example, we want a new authorized view on Google BigQuery and the SQL is stored in a file.
Reference file
The file could be just a simple SQL like this.
1
2
3
select 1 as id
union all
select 2 as id
Terraform file
In Terraform, we can add the file reference as below.
1
2
3
4
5
6
7
8
resource "google_bigquery_table" "sample_view" {
dataset_id = var.dataset
table_id = var.table
view {
use_legacy_sql = false
query = file("${path.module}/sample_query.sql") # file reference
}
}
${path.module}returns the path of the current “module”, which in this case it’d be current folder of this terraform resource file. Read more at Filesystem and Workspace Info.
Plan result
Terraform will read the file and stream to a string correctly.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Terraform will perform the following actions:
# google_bigquery_table.sample_view will be created
+ resource "google_bigquery_table" "sample_view" {
...
+ view {
+ query = <<-EOT
select 1 as id
union all
select 2 as id
EOT
+ use_legacy_sql = false
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
EOTstands for “End of Transmission”. This is a character to indicate the end of message.
<<-EOTimplies that starting here until foundEOTand call it a block of message.
templatefile()
Similar to file() function above, this function templatefile() reads files and also substitutes values. It’s very useful for a flexible design when we want to reuse assets multiple times.
Reference file
For example, the query requires several variables. We put each of them in format ${<variable_name>}.
1
2
3
select id, name
from `${project_id}.${dataset_id}.${table_id}`
where id in ${id_filter_list}
The template file syntax needs to follow Terraform string template. There are variables (${...}), if-statements (%{if ...}/%{else}/%{endif}), or for-loop (%{for ... in ...}/%{endfor}) allowed.
Terraform file
In Terraform, using function templatefile("<filepath>", "<map of variables>") as below.
1
2
3
4
5
6
7
8
9
10
11
12
13
resource "google_bigquery_table" "sample_view" {
dataset_id = var.dataset
table_id = var.table
view {
use_legacy_sql = false
query = templatefile("${path.module}/sample_querytemplate.sql.tmpl", {
project_id = "test_project",
dataset_id = "test_dataset",
table_id = "test_table",
id_filter_list = jsonencode([1, 2, 3])
})
}
}
jsonencode()returns JSON format of the parameter. In this case it will be a string “[1, 2, 3]” instead of an array [1, 2, 3].
Plan result
When plan we can see the results as expected. All variables have been compiled properly.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Terraform will perform the following actions:
# google_bigquery_table.sample_view will be created
+ resource "google_bigquery_table" "sample_view" {
...
+ view {
+ query = <<-EOT
select id, name
from `test_project.test_dataset.test_table`
where id in [1,2,3]
EOT
+ use_legacy_sql = false
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
These functions file() and templatefile() are very convenient when dealing with various file types plus we can improve file structure and decouple with the functions.
References
- file - Functions - Configuration Language | Terraform | HashiCorp Developer
- templatefile - Functions - Configuration Language | Terraform | HashiCorp Developer
- Filesystem and Workspace Info | References to Values - Configuration Language | Terraform | HashiCorp Developer
- jsonencode - Functions - Configuration Language | Terraform | HashiCorp Developer
- Strings and Templates - Configuration Language | Terraform | HashiCorp Developer
- End-of-Transmission character - Wikipedia
- google_bigquery_table | Resources | hashicorp/google | Terraform | Terraform Registry