opensource.google.com

Menu

Docs

Third Party METADATA

go/thirdparty/metadata

METADATA file

The METADATA file is intended to capture all third-party metadata various teams use to track and analyze third-party packages. All code within //third_party is required to have a METADATA file with a third_party field in the top-level directory of the package.

It’s occasionally useful to include extra information along with the package that falls outside of third-party metadata—e.g., instructions on how to build the package, a list of local modifications made to the package, etc. While the metadata format is designed to be flexible enough to handle all use cases, it’s not suitable for free-form text.

For those teams that need extra information, you can place it in a Markdown file (README.md, README.google.md, etc.), in a g3doc subdirectory, etc., which best aligns with your use case.

The third_party metadata field

TIP: Place the third_party field near the top of the METADATA file. That way people can quickly view third-party metadata.

The third-party metadata resides in a third_party field in the METADATA file. The normal required fields are: name, description, url, version, and last_upgrade_date. The url field identifies where it came from: ARCHIVE, GIT, SVN, HG, PIPER, EMAIL, or LOCAL_SOURCE.

NOTE: A PIPER URL doesn’t need the version and last_upgrade_date fields. A LOCAL_SOURCE URL doesn’t need the last_upgrade_date field.

Information about security and multiple versions is also encoded in the METADATA file. (See the third-party metadata.proto file below for information on how to fill out all fields in third_party.)

Example:

name: "jQuery"
description:
    "jQuery is a fast, small, and feature-rich JavaScript library. It makes "
    "things like HTML document traversal and manipulation, event handling, "
    "animation, and Ajax much simpler with an easy-to-use API that works "
    "across a multitude of browsers."

third_party {
  url {
    type: HOMEPAGE
    value: "https://jquery.com/"
  }
  url {
    type: GIT
    value: "https://github.com/jquery/jquery"
  }
  version: "2.2.2"
  last_upgrade_date {
    year: 2014
    month: 10
    day: 20
  }
  multiple_versions {
    type: TEMPORARY
    expiration {
      year: 2017
      month: 1
      day: 1
    }
    approval: "http://linkremoved/"
  }
}

Generate one for your package:

See go/thirdparty/metadata-generator.html

The last_upgrade_date metadata field

The security team uses the last_upgrade_date field to determine when the last sync to upstream was performed. With that information, they can determine if a security fix was applied to the source. Therefore, the last_upgrade_date should not be updated if the change is a local modification. Change this field only when upgrading the code from upstream.

The url metadata field

It’s important that we can identify the precise URL the code was retrieved from so that, among other things, the security team can correlate it with any known security vulnerabilities. The URL field has a lot of flexibility in order to accommodate the many different ways third-party packages are retrieved.

This is a list of examples of each type of URL to clarify some of their uses:

  • HOMEPAGE—The homepage for the package. This is optional but encouraged to help disambiguate similarly named packages or to get more information about the package. This is especially helpful when no other URLs provide human readable resources (such as git:// or sso:// URLs).

    url {
      type: HOMEPAGE
      value: "https://jquery.com/"
    }
    
  • ARCHIVE—The URL containing the source code for the specific version of the package.

    url {
      type: ARCHIVE
      value: "http://linkremoved/"
    }
    
  • GIT—The upstream git repository. This requires that the “version” field value specifies a specific git tag or revision.

    url {
      type: GIT
      value: "git://git.kernel.org/pub/scm/libs/netlink/libnl.git"
    }
    version: "2.0 20-Aug-2009"
    
    url {
      type: GIT
      value: "https://github.com/tensorflow/tensorflow"
    }
    version: "v0.9.0"
    

    Similarly for SVN and HG URLs.

  • PIPER—Primarily used when a package is being migrated into third_party from elsewhere in piper:

    url {
      type: PIPER
      value: "http://linkremoved//"
    }
    

    When a package is being newly developed in third_party, the PIPER URL should reference the package itself.

    url {
      type: PIPER
      value: "http://linkremoved/"
    }
    

    NOTE: The version and last_upgrade_date fields are not necessary for PIPER URLs.

  • EMAIL—The source code was received via email, or in some out-of-band way. This is almost always used with commercial software received directly from the vendor. In this case, the URL value should be used to provide additional information about how it was received:

    url {
      type: EMAIL
      value: "Delivered by Hedwig via Owl Postal Service"
    }
    
  • LOCAL_SOURCE—The location of the local copy of the package source code. This is typically only needed for java packages in google3 that build from source (go/thirdpartyjava):

    url {
      type: LOCAL_SOURCE
      value: "http://linkremoved/"
    }
    

    Also, packages on Git-on-Borg that import the full project history:

    url {
      type: LOCAL_SOURCE
      value: "https://android.googlesource.com/platform/external/apache-http"
    }
    

    NOTE: The last_upgrade_date field is not necessary for LOCAL_SOURCE URLs.

The updatemd tool

To help keep the METADATA file up-to-date, you can use the updatemd program to update commonly changed fields. For example:

$ /path/to/.../updatemd --help
Usage: updatemd [options] path

  -archive_url=""     : update the third-party "archive_url" field
  -print=""           : print the value of the field
  -version=""         : update the third-party "version" field

Except as otherwise noted, the content of this page is licensed under CC-BY-4.0 license. Third-party product names and logos may be the trademarks of their respective owners.