opensource.google.com

Menu

Docs

Distributing Project Artifacts

Once you’ve published your project, you may want to publish artifacts such as source code archives or pre-compiled binaries when you cut a new stable release. Additionally, many communities have specific distribution channels where projects or libraries are expected to be found.

Source code archives

Both GitHub and go/gob support auto-generated archives of a repository’s source code at a given ref. Simply by constructing the URL, you can link users to a downloadable archive of your project source code. Note that these only contain the source at that ref, not the full history of the repository. For example, the 1.6.2 release of Go is tagged “go1.6.2” in git, and so can be retrieved at any of:

GitHub Releases

If you need to ship additional artifacts as part of a release such as compiled binaries or documentation, you can use GitHub Releases. In addition to the auto-generated source code archives noted above, you can can provide release notes as well as binary attachments as part of release. As an example, see the google/protobuf releases page. Note that, at the time of writing, these attachments are limited to 2 GB each.

Google Cloud Storage

For projects not hosted on GitHub or that need support for larger files than GitHub allows, we recommend Google Cloud Storage. This is strongly recommended over Google Drive so that ownership of the files is not tied to a single team member’s account. For example, the Go team uses GCS for their binary releases on http://linkremoved/

Package repositories

Many communities have central repositories of packages and libraries, and Google has an official presence on many of them. The sections below provide instructions for releasing in these various repositories and who to contact for help or to get access.

If you have any questions, contact emailremoved@.

Debian

If you want to provide source code to Debian, for the Debian project to compile and package a binary artifact that it distributes via apt, please follow this process. You may only make contributions to “main” or “contrib”. Never “nonfree”.

  1. Make sure the source you want to release has gone through go/releasing first.
  2. Make sure you are not distributing source to Debian that Google does not own, for example vendored third_party dependencies. If you are, please email emailremoved@, citing this policy, and explain. If your binary has third-party compile-time dependencies that the Debian project will pull, that is fine and does not require review.

Bintray

Account: https://bintray.com/google
Contact: emailremoved@

Bintray is a download center with REST automation & support for RPM & Debian packages. It is often used to distribute Android libraries via Maven or/and jCenter repositories.

Here are the detailed steps to distribute your Android library via Maven or/and jCenter:

  1. Create and setup your Bintray account

    • Log into your corporate Bintray account

    Note: If you don’t have an account yet, ask emailremoved@ for an invitation. You will receive a “Bintray - invitation” email.
    Just click on “Join the organization” link there and fill up your account data (using your corporate {{USERNAME}}@google.com address).

    Note: If you don’t have permissions to create repos under ‘google’ domain in Bintray, ask emailremoved@ for help.

    • Click on “Add a new repository” button to create your repo

      • Private access could be used while you are preparing the final release in order to hide it from non-authorized users. But to download and use the package, your username and private API key should be included inside ‘local.properties’ file linked to your sample app’s ‘build.gradle.’
        When you are ready to open your package to the World, change an access to Public.

      • Name (aka <repo_name>) is an important part, since it can’t be changed in future. And it will be the first part of your repository’s public url.
        E.g.: https://bintray.com/google//<your_library_package>

      • For Type just select “Maven”

      • Check the option: “GPG sign uploaded files using Bintray’s public/private key pair”.

    • Then you can click on “Add new package” button in order to fill up all the package details via web interface.
      Or you can just specify them inside your ‘build.gradle’ file during Step 2.

  2. Prepare your Android project

    • Add necessary dependencies:

      // build.gradle
      dependencies {
        // You probably already have this:
        classpath 'com.android.tools.build:gradle:2.2.0'
        // So just add bintray and maven plugins:
        classpath 'com.jfrog.bintray.gradle:gradle-bintray-plugin:1.6'
        classpath 'com.github.dcendents:android-maven-gradle-plugin:1.4.1'
      }
      

      Note: change version codes to the latest ones that are available.

    • Apply corresponding plugins:

      // build.gradle
      apply plugin: 'com.github.dcendents.android-maven'
      apply plugin: 'com.jfrog.bintray'
      
    • Include “install” and “bintray” targets

      • Here is the detailed guide with examples from Bintray: https://github.com/bintray/gradle-bintray-plugin/blob/master/README.md

      • Here is the complete ‘build.gradle’ from them: https://github.com/bintray/bintray-examples/blob/master/gradle-bintray-plugin-examples/android-maven-example/build.gradle

      • My minimal set of changes (I didn’t use a github repository for this project):

        // local.properties:
        bintray.user={{USERNAME}}@google
        bintray.apikey=<your_api_key>
        user.id={{USERNAME}}
        user.email={{USERNAME}}@google.com
        
        
        // build.gradle:
        Properties properties = new Properties()
        properties.load(project.rootProject.file('local.properties')
                .newDataInputStream())
        
        
        bintray {
           user = properties.getProperty("bintray.user")
           key = properties.getProperty("bintray.apikey")
           configurations = ['archives']
           pkg {
               repo = '<repo_name>'
               name = '<package_name>'
               userOrg = 'google'
               desc = '<description_of_your_library>'
               publicDownloadNumbers = false
           }
        }
        install {
            repositories.mavenInstaller {
                pom.project {
                    name '<your_library_name>'
                    description '<description_of_your_library>'
                    packaging 'aar'
                    developers {
                        developer {
                            name 'google'
                            id properties.getProperty('user.id')
                            email properties.getProperty('user.email')
                        }
                    }
                }
            }
        

        }

      • Add javadoc and source tasks if you wish your library’s users are able to access them.
        E.g.:

        // build.gradle
        task sourcesJar(type: Jar) {
            from android.sourceSets.main.java.source
            classifier = 'sources'
        }
        task javadoc(type: Javadoc) {
            source = android.sourceSets.main.java.source
            ext.androidJar = "${android.sdkDirectory}/platforms/${android.compileSdkVersion}/android.jar"
            classpath += (project.files(android.getBootClasspath().join(File.pathSeparator))
            + files(ext.androidJar))
            options.links("http://linkremoved//");
            failOnError false
        }
        afterEvaluate {
            javadoc.classpath += files(android.libraryVariants.collect
            { variant ->
                variant.javaCompile.classpath.files
            })
        }
        task javadocJar(type: Jar, dependsOn: javadoc) {
            classifier = 'javadoc'
            from javadoc.destinationDir
        }
        artifacts {
            archives javadocJar
            archives sourcesJar
        }
        
  3. Install and upload your library into Bintray

    • Just open the ‘Terminal’ tab inside Android Studio or anywhere else and type:

      ./gradlew build install bintrayUpload --info
      
    • Check that your library and files are now inside your Bintray’s repo via web interface

  4. Test that everything works from your sample app

    • In order to include your library into an Android sample app, just update ‘build.gradle’ of it with the following lines:

      • For Public repos:

        // build.gradle
        repositories {
            maven {
                url "http://linkremoved/>"
            }
        } dependencies {
            compile '<your_package>:<library_name>:<version>'
        }
        
      • For Private ones:

        // build.gradle
        repositories {
            maven {
                url "http://linkremoved/>"
            }
            credentials {
                username '{{USERNAME}}@google'
                password '<your_API_key>'
            }
        } dependencies {
            compile '<your_package>:<library_name>:<version>'
        }
        
    • Then just try to use some classes and methods from your library:

      • Verify that sample app compiles without errors
      • Check that Android Studio correctly shows the javadoc.
  5. Add your library to jCenter (optional)

    • If you wish to distribute your library via jCenter as well as via Maven, then you need to:

      • Check if there is a button “Add to JCenter” in the lower right corner of your package’s Bintray page: https://bintray.com/google//<your_package>

        • Click it if you see the button
      • Otherwise, write an email to emailremoved@ about your project and ask him to enable jCenter distribution for your package

Useful resources from 3rd party developers:

  • How to distribute your own Android library through jCenter and Maven

Maven (Java)

http://repo1.maven.org/maven2 is the standard maven repository, available by default to any maven build, and indexed at http://linkremoved/. The direct way to publish an artifact into this repository is to go through http://linkremoved/.

Some example release steps can be found in the Dagger and AutoValue release docs.

Note: This assumes knowledge of how to use mvn deploy to push a binary to a repository to which one has authority. General maven instructions can be found in the maven book and help can be found at emailremoved@. If your project uses Gradle to build, you can take a look at ZetaSketch’s release guide and build script as an example.

oss.sonatype.org

Maven artifacts are “deployed” to a release repository. Open source projects can publish their released binaries to oss.sonatype.org, which will create an ad-hoc staging repository to which one deploys a release. The release engineer will then use the dashboard on oss.sonatype.org to close and release that staging repository, at which point it will be published to the main repository mirrors.

The following includes non-sonatype actions, as a full release typically involves some intertwined steps.

Note: Maven artifacts are write-once, so don’t close and release unless you are certain your artifact is correct.

Setup

As a one-time setup, pushing to sonatype requires:

  1. an account on oss.sonatype.org (sign up on http://linkremoved/)
  2. permission for that account to push artifacts with the group com.google.<yourpackage> (file an issue on http://linkremoved/)
  3. a pgp certificate (via gnupg) with a public key published to pgp.mit.edu via gpg --keyserver hkp://pool.sks-keyservers.net --send-keys <yourkeyid> or the equivalent
  4. your project must inherit from the sonatype oss parent artifact (see Dagger parent pom.xml here as an example), or manually declare a release repository with the id sonatype-nexus-staging
  5. a ${HOME}/.m2/settings.xml file containing the credentials for the account created in step #1, associated with the server id sonatype-nexus-staging. <<— important
  6. [optional] Create a release profile to make releases a one-line operation. e.g. protobuf-testing which has a custom release profile.
    • Projects which inherit from org.sonatype.oss:oss-parent will get such a profile automatically, called sonatype-oss-release

The administrative steps above are all documented in Sonatype's OSSRH Guide. The GPG instructions particular to this process can be found in this Sonatype GPG blog entry.

Regular releasing steps

Note: For concrete examples of workflows, see Dagger or AutoValue release instructions. These use pre-fab deployment scripts. Some of the steps below are amenable to automation.

For each release, you would:

  1. Ensure that the project does not depend on any projects with a -SNAPSHOT version number.
  2. Update your project’s pom.xml files to the intended release version.
  3. Perform a clean build and test of your artifacts
  4. Perform any source-repository-related maintenance
    1. tag the commit and push the tag to github, etc.
    2. prepare release notes, update release entry in github, etc (can be in parallel)
  5. Generate the artifact, javadoc, and sources sub-artifacts, and sign them.
    • Generally people set up a release profile a mvn -Prelease deploy (or mvn -Psonatype-oss-release deploy if using sonatype’s parent pom.xml)
  6. Log into oss.sonatype.org and
    1. Navigate to the staging repositories
    2. Find the staging repository for your release (nearly always the last)
    3. Spot-check the contents, the version number, etc.
    4. “Close” the repository in preparation for release.
    5. Hit refresh a lot (approx 1 minute wait time)
    6. “Release” the repository to sonatype’s mirrors.
  7. Announce on any relevant media.

Alternatives

TODO(**********) describe bintray->sonatype and other links.

npm (JavaScript)

Account: https://www.npmjs.com/org/google
Contact: emailremoved@

npm is the package repository for JavaScript packages for Node.js and browser environments.

Here are the basic guidelines to publishing packages for Google-owned code to npm:

  • As a first step, please review the OSPO guidelines on creating and releasing code publicly (go/releasing).

  • Figure out a good name and namespace (scope) for your module. npm modules can be published in the global namespace (e.g. eventid) or they can be published under the a Google namespace (e.g. @google/maps, or @google-cloud/vision). We recommend publishing under the @google and @google-cloud scopes for modules that have tie-ins with official Google products. Other modules that have general applicability to JavaScript code, but aren’t necessarily related to Google products can be published in the top level namespace. Some examples:

    • @google/maps is an API for Google Maps. It makes sense to publish it under the @google scope.
    • eventid on the other hand is a generally useful module, not related to any specific Google product. It makes sense for it to be in the global namespace.
  • Published modules should be owned by one of the Google organizations. This helps ensure that we retain admin control over the project in case you lose access to the module. Note that on npm, organizations and scopes (also called namespaces) are distinct concepts. You can publish a globally scoped module that is part of the @google npm organization.

  • To publish a new npm package for code hosted on GitHub, you can use our self-serve publishing tool hosted at http://linkremoved/

    • The tool uses your GitHub account to authenticate and validate that you have push access to the repo and then grants you a publishing token.
    • A bot manages the actual publishing to npm for you. It eliminates the need for you to manage npm accounts and teams, and deal with their security and 2FA configuration.
  • If the self-serve publishing tool is not suitable for your use-case; send an email to emailremoved@ with the following information:

    • The name of the package you want to publish and under which scope (global, @google, @google-cloud).
    • A ‘team name’ for the team of Googlers who will be managing the module (i.e. will have publish rights). There are two options to setup the team:
      1. Everyone who needs to manage the package already has an npm account. Please send us the list of the npm accounts to add to the team.
      2. Use a shared team account. Please create a new account on npm with an appropriate team name and save credentials in Google’s password storage system.
    • Links to github repo (if any) and Ariane launch.
    • We will create a team for you and you will be able to publish/manage the module yourself.
  • Activate two-factor authentication (2FA) for auth and writes on your npm account. npm supports 2FA, although the standard-issue Google security keys are currently not supported and you must use apps like Google Authenticator. Given the well-known security considerations with npm, we strongly recommend 2FA for all Googlers who publish Google source code to npm.

    When enabling 2FA, make sure to save the recovery codes for your account. We do not have the ability to recover your npm account if you get locked out.

    For npm modules that are managed by a team, we recommend everyone on a team needing to publish modules to have their own npm account. Presently npm doesn’t provide an elegant way to support 2FA codes for a shared team account.

  • Do not create new namespaces or organizations for Google projects. If you think you need a new namespace or org, please contact emailremoved@ first.

  • Feel free to contact emailremoved@ if you have any questions!

Packagist (PHP)

Account: https://packagist.org/packages/google/

TODO

PyPI (Python)

  1. Please review the OSPO guidelines on creating and releasing code publicly (go/releasing).
  2. Figure out a good name for your project. Python packages all share a global namespace, so pick a good name that isn’t taken by another package or would be confused with another package.
  3. Read through the general tutorial on packaging python projects.
  4. If you questions or issues creating your setup.py or your project has complex distrubtion needs reach out to emailremoved@ and emailremoved@ for assistance.
  5. If your packages wants to use the google import namespace (for example if you want your package to be import google.something) you must reach out to emailremoved@ and emailremoved@ to coordinate. Releasing a broken namespace package will break Python installations.
  6. Create an account on pypi.org specifically for your team or package. Store the password in go/valentine and share it with your team.
  7. Upload your package as that account.
  8. Once your package is uploaded use the pypi.org management interface to add google_opensource as a owner on the package.
  9. You may add additional accounts to the package as needed, but do not remove the google_opensource account.

RubyGems

TODO

WordPress plugins

Account: http://linkremoved/

TODO

TODO(**********): mention go/code-signing-certificates in relevant place

Except as otherwise noted, the content of this page is licensed under CC-BY-4.0 license. Third-party product names and logos may be the trademarks of their respective owners.