This page documents rules for adding third-party Go code to //piper/third_party/golang.

IMPORTANT: Read go/thirdparty first. When creating a new directory, set *********************************** in the CL. For Go-specific questions, email emailremoved@.

What it’s for

The third_party/golang directory is for code that is developed outside of Piper and imported into Piper using the //piper/third_party/golang/update.go tool. The purpose of this tree is to enforce common standards about how new versions of Go code are imported into Piper.

Open source code that is developed with Piper as the source of truth (and possibly exported) does not need to adhere to the requirements of the third_party/golang tree and should generally live elsewhere in third_party.

Code which is being temporarily developed in Piper with the intention of an eventual move to an external repository may be added to third_party/golang.

Go requirements

Go code checked in to third_party/golang must compile using the current google3 Go release (using blaze).

To add a new project, you must use //piper/third_party/golang/update.go. Add an entry in the goPackage map. If update.go is problematic, you’ll either have to fix or modify update.go, ask emailremoved@ to change it for you, or request an exception to forego using update.go. (It is very unlikely that we will permit you to forego using update.go.) The update.go directory name should not start with “go”. If upstream is “gofoo”, just call it “foo” if foo is unique enough. If there are many foos, you can import “” as “barfoo”. Groups of closely related packages are sometimes imported as “bar/foo”, though this is not typical.

After running update.go, you must not make any manual changes (except to BUILD files, in the case of glaze getting confused by package main files in a go_binary making too many go_binary rules). But running update.go repeatedly should not make any changes to your CL. Any changes that update.go makes to the upstream sources are considered canonical, and exempt from the usual go/pristinecopy policy to check in unmodified copies of the upstream source first. When sending out a CL importing a new project, explicitly state in the CL description that you tried running update.go again to ensure there were no further changes.

If you need to exclude certain files or folders from repository, use a filters file in a “patches” directory. See similar cases.

The filters file a simple file with the following syntax:

  • Comments are any lines that matches ^\s*#
  • Exclude rules are any lines that matches ^exclude\s(.*) where the capture group is a regular expression used to exclude certain paths.
  • Include rules are any lines that matches ^include\s(.*) where the capture group is a regular expression used to include certain paths.

The filter is applied by evaluating every path in a repository the following way:

  • Paths are always rooted at the top of the repository with a leading ‘/’.
  • Paths end with a trailing ‘/’ if and only if referencing a directory.
  • The series of rules are evaluated in the order defined in the filters file until the first match is found, resulting in either an exclusion or inclusion.
  • There is an implicit “include all” rule at the bottom of every filter chain.

If you need to patch something, use a patch file in a “patches” directory. See similar cases. The update.go tool first pulls upstream, and then applies local patches. After adding a patch, it’s often a good idea to run update.go again to ensure that the patch applies cleanly and that tests still pass. Do not add patches for BUILD files. Try not to add patches at all if you can help it. It’s preferred to get your modification upstream first, and then pull it into google3.

One note on blaze test vs “go test“: blaze test runs tests with a different working directory. “go test” always sets the working directory to the directory of the test. To work around this and let Go tests still find e.g. their “testdata” directories, either:

  • Add an import load("//third_party/golang:third_party_go_test.bzl", "third_party_go_test") to your BUILD file and use a third_party_go_test target (with parameter glaze_kind = "go_test") instead of go_test.
  • Add a file “google_init_test.go” to directories that need it (see examples). If a file begins with “google_” then update.go ignores it (doesn’t delete it).

Notes on specific files required by the Google-wide third_party policy:

  • OWNERS: create this yourself. The update.go tool won’t, but also won’t touch it.
  • BUILD file’s licenses section: create this yourself. You can do it after update.go creates the BUILD file.
  • METADATA file’s description: fill this in yourself. Also fill in the “security” section if required.

Getting a review

To declare that you know the rules on this page (or at least to get a faster review), make your CL description like:

Import as third_party/golang/foo

No manual changes. Running update.go --dir=foo doesn’t modify anything.

Checked that blaze {build,test} third_party/golang/foo/... works.

Except as otherwise noted, the content of this page is licensed under CC-BY-4.0 license. Third-party product names and logos may be the trademarks of their respective owners.