opensource.google.com

Menu

Docs

Haskell

go/thirdpartyhaskell

This describes Haskell-specific guidance for checking code into //piper/third_party/haskell.

IMPORTANT: Read go/thirdparty first.

NOTE: The top level BUILD file is not required.

Determining which third-party packages have already been installed in google3

Some versions of some third-party Haskell packages are installed as part of the Haskell platform. Haskell packages that are not part of the platform are kept in subdirectories of //piper/third_party/haskell. This shows them all:

ls third_party/haskell/ghc/v8_0_2_k8/lib/ghc-8.0.2/ third_party/haskell

Using third-party packages that have already been installed

Third party Haskell packages are just as easy to use as any other Haskell library. You need to add a dependency to your google3 BUILD rule in the normal way. Your BUILD file may look something like this:

haskell_binary(
    name = "Main",
    srcs = ["Main.hs"],
    deps = ["//third_party/haskell/hslogger:hslogger"]
)

and your program will just do the obvious:

import System.Log  -- Will be coming from hslogger

If your program needs a specific version of a third_party package, then it can depend on targets from that version explicitly, as follows:

haskell_binary(
    name = "Main",
    srcs = ["Main.hs"],
    deps = ["//third_party/haskell/hslogger/v1_0_7:hslogger"]
)

However, it is recommended that the non-versioned targets be used wherever possible, for reasons outlined in go/thirdparty.

Installing and upgrading third-party packages

Automated installation

Most of the following procedure can be carried out automatically by the cabal2build tool. However, some manual intervention will still be needed before the result is ready to be checked in, so it’s worth reading on.

Make sure you have the cabal-install package on your system (and it has the package manifest up to date):

sudo apt-get install cabal-install
cabal update

Obtain cabal2build as follows:

blaze build -c opt //haskell/Google/Google3/Tools/Cabal2Build:cabal2build

After that, read the manual like this:

blaze-bin/haskell/Google/Google3/Tools/Cabal2Build/cabal2build --help | less

Usual invocation looks like this:

blaze-bin/haskell/Google/Google3/Tools/Cabal2Build/cabal2build \
    --fetch hslogger

As the final step of package import cabal2build should generate a corresponding versionless package (see below for motivation).

Dependencies

If your package X depends on another third-party package Y, install Y at the top level //third_party/haskell/Y instead of trying to put it as a subdirectory inside //third_party/haskell/X. A cabal2build invocation with --recursive does this automatically. Caveat: if you use --recursive, it is harder to split up the imports to separate CLs. So you probably want to import the packages one by one, creating a new CL after each of the invocations.

Internal patching

If patching is required to make the code compile or run properly, then it is preferred to submit the unchanged code first, and make the changes in a follow-up CL. Please add a note about this in the CL description.

If the code doesn’t even compile, it should be submitted with the build rules commented out and a TODO note to make it compilable and reenable.

This is most often needed for tests, which may make assumptions about test data file locations (you would need to set these up as data dependencies in the build rules, and use Google helpers to resolve the location at runtime). See //piper/…/FindFiles.hs.

Reviewers

Both emailremoved@ and emailremoved@ will automatically get added to your CL. We require a sort of “community policing” for language-specific third-party package domains, given that emailremoved@ cannot be fluent in all of these programming languages.

Upgrading an existing package

When upgrading an existing package to a newer version, if the license does not change, then you may submit the CL after approval from a Haskell-team member. You do not need to wait for approval from a member of ****************** as well. For the convenience of your reviewer(s), you may add a note like “Package upgrade only. No license changes.”

If it is convenient, you may group together multiple related package upgrades into a single CL.

Adding a new package

Every new package must be added in its own CL. No other changes should be included in the CL.

Typical import from Cabal

Typical directory structure of a Cabal package imported from Hackage should look like this:

third_party/haskell/hslogger/
  BUILD
  OWNERS
  v1_2_10/BUILD
  v1_2_10/LICENSE
  v1_2_10/METADATA
  v1_2_10/src/BUILD
  v1_2_10/src/System/Log.hs
  ...

The version level v1_0_7/BUILD file exports the license file and describes how to build the package:

# Description: hslogger-1.2.10
#
# hslogger is a logging framework for Haskell, roughly similar to
# Python's logging module.
#
# hslogger lets each log message have a priority and source be associated
# with it.  The programmer can then define global handlers that route
# or filter messages based on the priority and source.  hslogger also
# has a syslog handler built in.
#
# Configured with Cabal flags:
#   buildtests: False
#   small_base: True

package(
    default_visibility = ["//third_party/haskell/hslogger:__pkg__"],
)

licenses(["notice"])

exports_files(["LICENSE"])

load(":package_description.bzl", "description")
load(
    "//tools/build_defs/haskell:cabal_package.bzl",
    "cabal_haskell_package",
)

cabal_haskell_package(description = description)

Please note the use of the cabal_haskell_package Skylark macro, which creates an expected haskell_library with an additional build_test pointing at it (see //piper/…/build_test.bzl?l=1 for details). This ensures that third-party Haskell libraries are included in a Presubmit project, preventing them from being broken accidentally.

The versionless BUILD file provides the same targets as the versioned google3 package:

haskell_library(
    name = "hslogger",
    deps = ["//third_party/haskell/hslogger/v1_2_10:hslogger"],
)

Usually Cabal packages include a Setup.hs file which has zero value when imported, hence please delete this file. The same goes for all other files which clearly have zero value (e.g. marked as obsolete or broken).

If a package includes a test suite, please attempt to use haskell_test target to wrap these tests into something Presubmit can test. This helps to avoid breakage of third-party Haskell packages by changing infrastructure.

Google3 package directories for Haskell package versions

Convert a Haskell package version suffix into a google3 package directory name by prepending v, and turning . or - into _ to make it valid. For example, hslogger-1.2.10 should be checked in in a directory called v1_2_10 (under //piper/third_party/haskell/hslogger, of course).

Multiple versions

Haskell allows multiple versions of the same package to coexist in the same binary as long as they are in different packages. Hopefully we will not need to have that, but it is possible.

Tests

If the original package has any tests, they should be exposed to blaze in the BUILD file.

Updating third-party packages

To upgrade to a new version of the package, you should create a parallel directory next to the existing one, then migrate all the existing users of the old version to the new one and remove the old one. If possible, remove the old version in the same CL that you add the new one. When you add a new version, you must commit to removing the old one or reverting the new within two weeks. See go/oneversion for an explanation.

Users that depend upon the versionless package can all be migrated at once, just by migrating the versionless package, as follows:

blaze build //haskell/Google/Google3/Tools:update_versionless_third_party_package
blaze-bin/haskell/Google/Google3/Tools/update_versionless_third_party_package \
    //third_party/haskell/hslogger/v1_0_7:hslogger

Versionless third-party Haskell packages: motivation

When upgrading a Haskell package that is installed in google3 under //third_party/haskell/${package}, it’s quite hard to migrate google3 packages that depend on it. That’s because the new version is a new google3 package (//third_party/haskell/${package}/v${YYY} instead of //third_party/haskell/${package}/v${XXX}) so all the dependencies have to be updated.

To make matters worse, it’s often the case that some of the dependencies are in components while others aren’t. When a package is in components, the upgrade process looks like:

  1. Check in the new version of the package, and add it to components;
  2. Move dependencies that are in components to use the new package version;
  3. Wait for a components release;
  4. Move dependencies that are not in components to use the new package version.

Unfortunately this means that, immediately after the components release has happened (between steps three and four), code in components is seen to be using the new package version while code not in components is still using the old version. This is likely to break builds.

This is eased by introducing targets like //third_party/haskell/${package}:${package}. Such a target provides a single point at which to switch versions. It lives in components if and only if any of its dependencies do (as the version-specific google3 packages do) so there aren’t any new version skew issues.

With a versionless package in place, the upgrade process looks like:

  1. Check in the new version of the package, and add it to components;
  2. Move all targets defined in the versionless package to refer to the new version;
  3. Wait for a components release.

Except as otherwise noted, the content of this page is licensed under CC-BY-4.0 license. Third-party product names and logos may be the trademarks of their respective owners.